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ABSTRACT 


Hybrid  human-system  interfaces  (HSIs)  result  from  the  combination  of  new  (e.g.,  digital)  and  traditional  technologies.  New 
demands  may  be  imposed  on  personnel  in  operating  and  maintaining  these  systems.  These  demands  may  result  from  many 
factors  including  the  characteristics  of  the  new  technologies,  the  characteristics  of  the  mixture  of  new  and  traditional 
technologies,  the  process  by  which  the  hybrid  HSI  is  developed  and  used,  and  the  way  in  which  personnel  are  prepared  to 
use  the  hybrid  HSI.  The  objective  of  this  study  was  to  develop  human  factors  review  guidance  on  the  processes  by  which 
hybrid  HSIs  are  developed,  implemented,  and  integrated  into  plant  operations.  A  characterization  framework  was 
developed  for  describing  the  key  characteristics  of  hybrid  HSIs  that  are  important  to  human  factors  engineering  (HFE) 
reviews.  Then,  the  research  studies,  HFE  processes,  and  guidance  related  to  system  development  and  modernization  were 
reviewed.  This  information  was  used  as  the  technical  basis  upon  which  we  developed  the  design  review  guidelines.  This 
guidance  applies  to  general  work  that  should  be  undertaken  and  factors  that  should  be  considered  in  designing  and 
implementing  hybrid  HSIs,  particularly  in  upgrading  existing  HSIs.  The  establishment  and  analysis  of  design 
requirements,  interface  design,  and  the  evaluation  of  the  final  system  are  addressed.  Issues  for  further  research  were 
identified  for  process  consideration  for  which  the  technical  basis  was  insufficient  to  support  guidance  development. 
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EXECUTIVE  SUMMARY 


The  Human-System  Interface  Design  Review  Guideline,  NUREG-0700,  Rev.  1  (O’Hara,  Brown,  Stubler,  Wachtel,  and 
Persensky,  1996)  was  developed  to  provide  human  factors  engineering  (HFE)  guidance  to  the  U.S.  Nuclear  Regulatory 
Commission  (NRC).  The  NRC  staff  uses  NUREG-0700  for;  (1)  reviewing  the  human-system  interface  (HSI)  design 
submittals  prepared  by  licensees  or  applicants  for  a  license  or  design  certification  of  a  commercial  nuclear  power  plant 
(NPP),  and  (2)  perfonning  HSI  reviews  undertaken  as  part  of  an  inspection  or  other  type  of  regulatory  review  involving  HSI 
design  or  incidents  involving  human  performance.  It  describes  those  aspects  of  the  HSI  design  review  process  that  are 
important  to  identifying  and  resolving  human  engineering  discrepancies  that  could  adversely  affect  a  plant’s  safety. 
NUREG-0700  also  has  detailed  HFE  guidelines  for  assessing  the  implementation  of  HSI  designs.  In  developing  NUREG- 
0700,  Rev.  1,  several  “gaps”  were  identified.  The  gaps  were  topics  for  which  there  was  an  insufficient  technical  basis  upon 
which  to  formulate  guidance.  One  gap  is  the  integration  of  advanced  HSI  technology  into  conventional  NPPs. 

The  NRC  is  currently  sponsoring  research  at  Brookhaven  National  Laboratory  (BNL)  to  address  this  gap  and  to  (1)  better 
define  the  effects  of  changes  to  plant  HSIs,  brought  about  by  the  application  of  digital  technology,  on  personnel  performance 
and  plant  safety,  and  (2)  develop  HFE  guidance  to  support  safety  reviews  should  it  be  necessary  to  examine  plant 
modifications  involving  a  safety-significant  aspect  of  HSIs.  The  research  led  to  the  identification  of  several  human 
performance  topics  (O’Hara,  Stubler,  and  Higgins,  1996).  The  topics  then  were  evaluated  for  their  potential  safety 
significance  (Stubler,  Higgins,  and  O’Hara,  1996).  One  topic  found  to  be  potentially  significant  to  safety  and  so  selected  for 
the  development  of  HFE  guidance  was  Design  Analysis,  Evaluation,  and  Implementation  of  Hybrid  HSIs. 

This  topic  addresses  the  need  to  consider  human  factors  issues  during  the  analysis,  design,  and  evaluation  of  upgrades.  Of 
particular  concern  are  changes  in  roles  and  functions  of  plant  personnel  brought  about  by  changes  in  the  HSI.  Another 
aspect  of  this  topic  is  the  need  to  consider  human  factors  when  implementing  technology  changes  (upgrades),  including  the 
installation  of  the  changes  and  their  incorporation  into  a  plant’s  operating  practices.  Important  factors  included  the  effects 
on  personnel  of  temporary  and  changing  HSI  configurations,  personnel  training,  and  personnel  acceptance  of  HSI  changes. 
Thus,  this  topic  encompasses  the  life  cycle  of  an  HSI  upgrade  from  its  initial  planning  through  design,  evaluation,  and 
installation.  The  topic  title  was  renamed  HSI  and  Plant  Modernization  Process  to  more  clearly  convey  this  scope. 

The  objective  of  this  study  was  to  formulate  HFE  review  guidance  for  NPP  upgrades  to  plant  systems  and  the  HSI,  based  on 
a  technically  valid  methodology  for  developing  guidelines.  The  term  upgrade  is  used  to  include  any  type  of  change, 
modification,  or  retrofit  made  to  HSI  components  or  plant  systems  that  may  influence  personnel  performance.  Several  tasks 
were  performed,  including  the  following: 

•  A  technical  basis  was  established  using  human  performance  research  and  analyses  of  upgrades, 

•  HFE  review  guidelines  were  developed  in  a  format  consistent  with  NRC  guidance,  and 

•  Remaining  issues  were  identified  for  which  research  is  insufficient  to  support  the  development  of  NRC  review 
guidance. 

The  status  of  each  is  briefly  discussed  below. 

Technical  Basis  Development 

The  effects  of  upgrades  on  personnel  performance  were  addressed  by  examining  basic  HFE  literature,  literature  on  complex 
human-machine  systems,  and  industry  experience  gained  from  site  visits,  interviews,  and  industrial  literature.  The 
resulting  technical  basis  described  human  performance  in  complex  human-machine  systems  in  terms  of  skilled  task 
performance  and  the  basic  properties  of  human  information  processing,  such  as  memory  and  attention,  supporting  it.  The 
types  of  knowledge  and  skills  that  must  be  adapted  to  a  new  work  design  after  an  upgrade  was  addressed. 
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These  human  performance  considerations  were  then  addressed  within  the  context  of  the  NRC’s  design  process  review 
guidance.  The  Standard  Review  Plan,  NUREG-0800  (NRC,  1996),  specifies  the  Human  Factors  Engineering  Program 
Review  Model,  NUREG-071 1  (O’Hara  et  al.,  1994),  as  the  guidance  for  HFE  reviews  of  design  processes.  NUREG-071 1 
provides  criteria  for  evaluating  the  incorporation  of  HFE  principles  and  practices  in  the  design  of  HSIs  in  NPPs,  and  for 
evaluating  the  implementation  of  the  final  design.  It  was  originally  developed  to  support  the  review  of  NPP  designs  for 
certification  under  10  CFR  Part  52.  However,  it  is  being  revised  and,  in  accordance  with  NUREG-0800,  will  provide  the 
criteria  for  all  HFE-related  staff  reviews,  i.e.,  applicable  to  Part  50  as  well  as  Part  52  applicants. 

A  central  tenet  of  NUREG-071 1  is  that  the  HFE  aspects  of  the  plant  should  be  developed,  designed,  and  evaluated  on  the 
basis  of  a  structured  system  analysis  using  accepted  HFE  principles.  NUREG-071 1  reflects  a  top-down  approach  for 
conducting  an  NRC  safety  evaluation  so  that  the  significance  of  individual  topics  may  be  seen  in  relationship  to  the  high- 
level  goal  of  plant  safety.  Top-down  refers  to  an  approach  starting  at  the  “top”  with  the  plant’s  high-level  mission  goals 
and  breaking  them  down  into  the  functions  necessary  to  achieve  the  goals.  Functions  are  allocated  to  human  and  system 
resources  and  are  split  into  tasks.  Personnel  tasks  are  analyzed  for  specifying  the  alarms,  information,  and  controls  that  will 
be  required  to  allow  them  to  accomplish  assigned  functions.  Tasks  are  arranged  into  meaningful  jobs  assigned  to  individual 
operators  and  the  HSl  is  designed  to  best  support  job  performance.  The  detailed  design  (of  the  HSI,  procedures,  and 
training)  is  the  “bottom”  of  the  top-down  process.  The  HFE  safety  evaluation  should  include  HFE  aspects  of  normal  and 
emergency  operations,  tests,  and  maintenance. 

NUREG-071 1  has  ten  design  process  review  elements: 

•  HFE  Program  Management 

•  Operating  Experience  Review 

•  Functional  Requirements  Analysis  and  Function  Allocation 

•  Task  Analysis 

•  Staffing 

•  Human  Reliability  Analysis 

•  Human-System  Interface  Design 

•  Procedure  Development 

•  Training  Program  Development 

•  Human  Factors  Verification  and  Validation. 

Each  element  is  divided  into  four  sections:  Background,  Objective,  Applicant  Submittals,  and  Review  Criteria.  The 
review  criteria  are  oriented  to  new  plant  designs.  Thus  guidance  to  address  the  unique  considerations  associated  with 
upgrades  had  to  be  developed. 
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HFE  Review  Guidelines 


Guidance  for  reviewing  the  design  process  aspects  of  upgrades  was  developed.  The  guidance  was  organized  according  to 
the  ten  review  elements  of  NUREG-071 1.  In  addition,  a  general  guidance  section  was  developed. 

Within  each  element,  guidance  was  further  organized  into  four  sections.  The  first  describes  the  conditions  under  which  the 
particular  NUREG-071 1  element  is  relevant  to  the  review  of  upgrades.  The  second  category  includes  modified  guidance 
from  NUREG-071 1  that  focuses  on  characteristics  and  considerations  for  upgrades.  The  guidance  in  the  third  category  is 
specifically  relevant  to  upgrades,  but  does  not  appear  in  NUREG-071 1.  The  fourth  category  has  considerations  that  have 
potential  applications  beyond  upgrades,  and  are  possible  additions  to  the  more  general  guidance  of  NUREG-07 1 1 . 

Upgrade  Issues  Requiring  Additional  Research 

Several  human  performance  issues  associated  with  upgrades  were  identified.  They  represent  topics  for  which  research  is 
necessary  before  more  guidance  can  be  developed: 

•  The  Effects  of  HSI  Inconsistency  on  Alternating  Use  of  HSI  Components 

•  The  Effects  of  HSI  Design  on  Crew  Coordination  and  Cooperation 

•  The  Role  of  Training  in  HSI  Skills 

•  The  Effects  of  the  Installation  Process  for  HSI  Upgrades  on  Personnel  Performzince 

•  Personnel  Acceptance  of  Upgrades. 


This  guidance  will  be  integrated  into  existing  NRC  review  guidance  documents  and  will  give  the  NRC  staff  the  technical 
basis  to  help  ensure  that  modifications  of  HSI  designs  do  not  compromise  safety.  Thus,  the  results  of  this  project  are 
expected  to  contribute  to  satisfying  the  NRC’s  goals  of  (1)  maintaining  safety,  (2)  increasing  public  confidence,  (3) 
increasing  regulatory  efficiency  and  effectiveness,  and  (4)  reducing  unnecessary  burden  on  personnel. 
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1  INTRODUCTION 


1.1  Background 

The  Human-System  Interface  Design  Review  Guideline  (NUREG-0700,  Rev.  1)  (O’Hara,  Brown,  Stubler,  Wachtel,  and 
Persensky,  1996)  was  developed  to  provide  human  factors  engineering  (HFE)  guidance  to  the  U.S.  Nuclear  Regulatory 
Commission  (NRC).  The  NRC  staff  uses  NUREG-0700  for:  (1)  reviewing  the  human-system  interface  (HSI)  design 
submittals  prepared  by  licensees  or  applicants  for  a  license  or  design  certification  of  a  commercial  nuclear  power  plant 
(NPP),  and  (2)  performing  HSI  reviews  undertaken  as  part  of  an  inspection  or  other  type  of  regulatory  review  involving  HSI 
design  or  incidents  involving  human  performance.  It  describes  those  aspects  of  the  HSI  design  review  process  that  are 
important  to  identifying  and  resolving  human  engineering  discrepancies  that  could  adversely  affect  a  plant’s  safety. 
NUREG-0700  also  has  detailed  HFE  guidelines  for  assessing  the  implementation  of  HSI  designs. 

In  developing  NUREG-0700,  Rev.  1,  several  topics  were  identified  as  “gaps”  because  there  was  an  insufficient  technical 
basis  upon  which  to  formulate  guidance.  One  such  topic  is  integrating  advanced  HSI  technology  into  conventional  NPPs. 
The  NRC  is  currently  sponsoring  research  at  Brookhaven  National  Laboratory  (BNL)  to  (1)  better  define  the  effects  of 
changes  to  plant  HSIs,  brought  about  by  the  application  of  digital  technology,  on  personnel  performance  and  plant  safety, 
and  (2)  develop  HFE  guidance  to  support  safety  reviews  should  it  be  necessary  to  examine  plant  modifications  involving  a 
safety-significant  aspect  of  HSIs.  This  guidance  will  be  integrated  into  NUREG-0700  and  will  give  the  NRC  staff  the 
technical  basis  to  help  ensure  that  such  modifications  of  HSI  designs  do  not  compromise  safety. 

The  results  of  this  project  are  expected  to  contribute  to  satisfying  the  NRC’s  goals  of  (1)  maintaining  safety,  (2)  increasing 
public  confidence,  (3)  increasing  regulatory  efficiency  and  effectiveness,  and  (4)  reducing  unnecessary  burden  on  personnel. 

Based  upon  the  literature,  interviews,  and  site  visits,  changes  in  HSI  technology  and  their  potential  effects  on  human 
performance  were  identified  (O’Hara,  Stubler,  and  Higgins,  1996).  The  topics  then  were  evaluated  for  their  potential  safety 
significance  (Stubler,  Higgins,  and  O’Hara,  1996).  One  topic  found  to  be  potentially  significeint  to  safety  and  so  selected  for 
the  development  of  HFE  guidance  was  Design  Analysis,  Evaluation,  and  Implementation  of  Hybrid  HSIs. 

This  topic  resulted  from  the  combination  of  two  separate  but  related  topics:  (1)  Design  Analysis  and  Evaluation  of  Hybrid 
HSIs,  and  (2)  Upgrade  Implementation  of  Hybrid  HSIs.  The  first  topic  addressed  the  need  to  consider  human  factors  issues 
during  analysis  and  evaluations  conducted  when  designing  upgrades.  The  objective  was  to  ensure  that  human  performance 
considerations  are  adequately  addressed  during  the  process.  Of  particular  concern  were  changes  in  people’s  roles  and 
functions  brought  about  by  changes  in  the  HSI.  The  second  topic  addressed  the  need  for  considering  human  factors  when 
implementing  technology  changes  (upgrades),  including  their  installation  and  incorporation  into  a  plant’s  operating 
practices.  Important  factors  included  the  effects  upon  personnel  of  temporary  and  changing  HSI  configurations,  personnel 
training,  and  personnel  acceptance  of  HSI  changes.  Thus,  combining  these  two  topics  resulted  in  one  larger  topic  that 
encompasses  the  life  cycle  of  an  HSI  upgrade  from  its  initial  planning  through  design,  evaluation,  and  installation.  The 
topic  title  was  renamed  HSI  and  Plant  Modernization  Process  to  more  clearly  convey  this  scope. 

In  this  report,  the  term  upgrade  is  used  to  describe  the  types  of  design  changes  addressed  by  this  review.  It  is  used 
generically  to  include  any  type  of  change,  modification,  or  retrofit  made  to  HSI  components  or  plant  systems  that  may 
influence  personnel  performance.  This  usage  is  consistent  with  that  in  industry,  as  in  the  Guideline  on  Licensing  Digital 
Upgrades  (EPRI TR- 102348)  from  the  Electric  Power  Research  Institute  (EPRI,  1993). 

1.2  HSI  and  Plant  Modernization  Processes 

The  topic  of  this  report  is  oriented  toward  the  design  process  for  upgrading  the  HSI  or  plant  equipment  rather  than  toward 
the  design  details  of  any  one  specific  technology,  such  as  soft  controls  or  computer-based  procedures.  Therefore,  it  is 
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important  to  understand  how  the  NRC  staff  generally  reviews  factors  in  the  HFE  design  process.  The  guidance  produced  in 
this  study  is  fully  consistent  with,  and  integrated  with,  the  review  methodologies  already  developed.  The  staffs  review 
methods  and  criteria  are  primarily  described  in  three  main  documents,  NUREGs  0800, 071 1,  and  0700.  In  addition,  several 
NRC  inspection  manuals  are  relevant.  To  provide  the  reader  with  a  better  understanding  of  the  guidance  developed  in  this 
document,  each  of  these  documents  is  briefly  described  below. 

1.2.1  Standard  Review  Plan  (NUREG-0800) 

The  Standard  Review  Plan,  NUREG-0800  G'fRC,  1996),  was  originally  published  in  1981  and  was  updated  in  1996  to 
reflect  the  staffs  and  industry’s  experience  with  safety  reviews.  Importantly,  one  reason  for  the  change  was  “...to  reflect  the 
experience  of  the  safety  reviews  conducted  on  design  certification  applications  for  evolutionary  nuclear  power  plants.”  The 
revised  Standard  Review  Plan  (SRP)  is  applicable  to  both  10  CFR  Part  50  and  10  CFR  Part  52  applicants,  thus  bringing  the 
review  of  existing  license  holders  and  applicants  for  advanced  plants  under  a  common  review  methodology. 

Chapter  18,  Human  Factors  Engineering,  discusses  the  review  of  the  HFE  aspects  of  NPPs.  The  purpose  of  the  HFE  review 
is  “...to  improve  safety  by  verifying  that  accepted  human  factors  engineering  practices  and  guidelines  are  incorporated  into 
the  program  design.”  The  review  covers  both  the  design  process  and  its  product. 

The  acceptance  criteria  are  divided  into  ten  broad  areas  of  review:  HFE  Program  Management;  Operating  Experience 
Review;  Functional  Requirements  Analysis  and  Allocation  Analysis;  Task  Analysis;  Staffing;  Human  Reliability  Analysis; 
Human-System  Interface  Design;  Procedure  Development;  Training  Program  Development;  and.  Human  Factors 
Verification  and  Validation.  The  SRP  provides  general  criteria  for  each  of  these  areas  of  review,  and  refers  to  NUREG-071 1 
and  NUREG-0700,  Rev.  1  (discussed  in  the  next  two  sections)  for  more  specific  details. 

As  the  SRP  in  general  has  broad  applicability,  the  methodology  is  tailored  to  the  needs  of  the  specific  review.  “For  any 
given  application,  the  staff  reviewers  may  select  and  emphasize  particular  aspects  of  each  SRP  section  as  is  appropriate  for 
the  application...  the  staff  may  not  carry  out  in  detail  all  of  the  review  steps  listed  in  each  SRP  section  in  the  review  of  every 
application”  (p.  3).  In  Chapter  18,  tailoring  of  the  review  is  identified  as  well.  The  SRP  notes  that: 

While  this  review  process  defines  10  areas  of  review,  not  all  may  be  applicable  to  reviewing  an  applicant’s  human  factors 
engineering  program.  Judgement  regarding  areas  of  review  to  be  given  attention  for  an  applicant’s  submittal  should  be 
based  on  evaluation  of  the  information  provided  by  the  applicant,  the  similarity  of  the  associated  HFE  issues  to  those 
recently  reviewed  for  other  plants,  and  the  determination  of  whether  items  of  special  or  unique  safety  significance  are 
involved.  Also,  the  relevance  of  each  area  of  review  and  the  appropriate  level  of  detail  of  evidence  should  be  considered 
with  respect  to  such  factors  as  the  purpose  of  the  review,  the  nature  of  the  HFE  concern,  the  status  of  the  applicant’s  design 
process,  the  scope  of  the  design,  and  the  goal  of  the  review  (e.g.,  design  certification,  licensing),  (p.  18.0.2) 

The  predominant  technical  basis  of  the  SRP  is  the  Human  Factors  Engineering  Program  Review  Model  (NUREG-071 1). 
This  document  is  discussed  in  the  next  section. 

1.2.2  Human  Factors  Engineering  Program  Review  Model  (NUREG-071 1) 

The  Human  Factors  Engineering  Program  Review  Model,  NUREG-071 1  (O’Hara  et  al.,  1994)  provides  criteria  for 
evaluating  the  incorporation  of  HFE  principles  and  practices  in  the  design  of  HSIs  in  NPPs,  and  for  evaluating  the 
implementation  of  the  final  design.  It  was  originally  developed  to  support  the  review  of  NPP  designs  for  certification  under 
10  CFR  Part  52.  However,  it  is  being  revised  and,  in  accordance  with  the  SRP,  will  provide  the  criteria  for  all  HFE-related 
staff  reviews,  i.e.,  applicable  to  Part  50  as  well  as  Part  52  applicants. 
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A  central  tenet  of  NUREG-071 1  is  that  the  HFE  aspects  of  the  plant  should  be  developed,  designed,  and  evaluated  on  the 
basis  of  a  structured  system  analysis  using  accepted  HFE  principles.  NUREG-071 1  reflects  a  top-down  approach  for 
conducting  an  NRC  safety  evaluation  so  that  the  significance  of  individual  topics  may  be  seen  in  relationship  to  the  high- 
level  goal  of  plant  safety.  Top-down  refers  to  an  approach  starting  at  the  “top”  with  the  plant’s  high-level  mission  goals 
and  breaking  them  down  into  the  functions  necessary  to  achieve  the  goals.  Functions  are  allocated  to  human  and  system 
resources  and  are  split  into  tasks.  Operator’s  tasks  are  analyzed  for  specifying  the  alarms,  information,  and  controls  that 
will  be  required  to  allow  the  operator  to  accomplish  assigned  functions.  Tasks  are  arranged  into  meaningful  jobs  assigned 
to  individual  operators  and  the  HSI  is  designed  to  best  support  Job  performance.  The  detailed  design  (of  the  HSI, 
procedures,  and  training)  is  the  “bottom”  of  the  top-down  process.  The  HFE  safety  evaluation  should  be  broad-based  and 
include  HFE  aspects  of  normal  and  emergency  operations,  tests,  and  maintenance. 

NUREG-071 1  has  ten  elements  organized  by  the  stages  of  planning,  analysis,  design,  and  verification  and  validation 
(Figure  1.1).  Each  element  is  divided  into  four  sections:  Background,  Objective,  Applicant  Submittals,  and  Review 
Criteria. 


Stages  of  the  HFE  Design  and 
Implementation  Process 


I 


Planning 


Element  1 
HFE  Program 
Management 


Analysis 

Element  2 

Operating  Experience 
Review 


Element  3 
Functional  Requirements 
Analysis  and 
Function  Allocation 


Element  4 
Task 
Analysis 


Element  5 
Staffing 


Element  6 
Human  Reliability 
Analysis 


Design 


Element  7 
Human-System 
Interface  Design 


Element  8 
Procedure 
Development 


Element  9 
Training  Program 
Development 


V&V 


Element  10 
Human  Factors 
Verification 
and  Validation 


Figure  1.1  Elements  of  Human  Factors  Engineering  Program  Review  Model  (NUREG-071 1) 
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•  Background-  A  brief  explanation  is  given  of  the  rationale  and  purpose  of  each  element. 

•  Objective  -  The  review  objective(s)  of  the  element  is  defined. 

•  Applicant  Submittals  -  Materials  to  be  provided  for  the  NRC’s  review  are  listed.  Generally,  three  reports  are 
identified:  implementation  plan  (the  applicant’s  proposed  methodology  for  meeting  the  acceptance  criteria  of  the 
element),  analysis-results  report  (the  results  of  the  applicant’s  efforts  on  a  NUREG-071 1  element  with  respect  to 
the  review  criteria),  and  design-team  review  report  (an  independent  evaluation  of  the  activities  addressed  for  the 
element  by  the  design  team).  In  addition  to  reports,  the  reviewer  may  require  samples  of  work  products  for  earlier 
elements,  and  implemented  designs  for  later  elements,  such  as  verification  and  validation  (V&V). 

•  Review  Criteria  -  This  section  contains  the  acceptance  criteria  for  design  process  products  and  for  the  final  design 
review.  Not  all  existing  NRC  detailed  criteria  for  the  final  design  are  duplicated.  For  example,  NUREG-0700 
contains  HFE  guidance  for  detailed  reviews  of  control  room  design.  NUREG-0700  is  only  referred  to  in  the 
applicable  NUREG-071 1  elements. 

NUREG-071 1  states  that  the  applicant  should  undertake  the  types  of  activities  described  for  each  element  using  accepted 
HFE  practices,  as  specified  by  applicable  regulatory  documents  and  HFE  codes,  standards,  and  guidelines.  Each  of  the 
NUREG-071 1  elements  lists  the  documents  that  may  be  used. 

A  brief  overview  of  the  focus  of  each  element  follows: 

Element  1  -  HFE  Program  Management 

The  overall  purpose  of  the  HFE  program  review  is  to  ensure  that 

•  The  applicant  has  integrated  HFE  into  the  plant’s  development,  design,  and  evaluation. 

•  The  applicant  has  provided  HFE  products  (e.g.,  HSIs,  procedures,  and  training)  that  make  it  possible  to  perform 
operation,  maintenance,  test,  inspection,  and  surveillance  tasks  safely,  efficiently,  and  reliably. 

•  The  HFE  program  and  its  products  reflect  “state-of-the-art  human  factors  principles”  [10  CFR  50.34(f)(2)(iii))  as 
required  by  10  CFR  52.47(a)(l)(ii)]  and  satisfy  all  specific  regulatory  requirements,  as  stated  in  10  CFR. 

The  objective  of  this  review  topic  is  to  ensure  that  the  applicant  has  an  HFE  design  team  with  the  responsibility,  authority, 
placement  within  the  organization,  and  composition  to  ensure  that  the  design  commitment  to  HFE  is  met.  Also,  the  team 
should  be  guided  by  a  plan  to  ensure  that  the  HFE  program  is  properly  developed,  executed,  overseen,  and  documented. 
This  plan  should  describe  the  technical  program  elements  ensuring  that  all  aspects  of  HSI  are  developed,  designed,  and 
evaluated  on  the  basis  of  a  structured  top-down  systems-analysis  using  accepted  HFE  principles. 

Element  2  -  Operating  Experience  Review 

The  main  purpose  of  conducting  an  operating  experience  review  (OER)  is  to  identify  HFE-related  safety  issues.  The  OER 
should  provide  information  on  the  past  performance  of  fully-integrated  predecessor  systems.  This  approach  is  analogous  to 
full-mission  validation  tests,  which  provide  information  about  the  achievement  of  HFE  design  goals  supporting  safe  plant 
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operation  for  the  integrated  system  under  review.  The  issues  and  lessons  learned  from  previous  operating  experience 
provide  a  basis  for  improving  the  plant  design  in  a  timely  way;  i.e.,  at  the  beginning  of  the  design  process. 

The  objective  of  reviewing  this  topic  is  to  assure  that  the  applicant  has  identified  and  analyzed  any  HFE-related  problems 
and  issues  in  previous  designs  that  are  similar  to  the  current  design  under  review.  In  this  way,  negative  features  associated 
with  predecessor  designs  may  be  avoided  in  the  current  one  while  retaining  positive  features.  The  OER  should  address  the 
predecessor  systems  upon  which  the  design  is  based,  selected  technological  approaches  (e.g.,  if  touch-screen  interfaces  are 
planned,  the  HFE  issues  associated  with  using  them  should  be  reviewed),  and  the  plant’s  HFE  issues  (e.g.,  generic  safety 
issues  defined  by  the  NRC). 

Element  3  -  Functional  Requirements  Analysis  and  Allocation 

The  purpose  of  this  element  of  the  review  is  to  ensure  that  the  applicant  has  defined  the  plant’s  safety  functional 
requirements  and  that  the  function  allocations  take  advantage  of  human  strengths  and  avoid  allocating  functions  that  would 
be  negatively  affected  by  human  limitations.  The  operator’s  role  is  examined  in  two  steps:  functional  requirements 
analysis,  and  function  allocation  (assignment  of  levels  of  automation). 

Functional  requirements  analysis  is  the  identification  of  those  functions  which  must  be  performed  to  satisfy  the  plant’s 
safety  objectives,  i.e.,  to  prevent  or  mitigate  the  consequences  of  postulated  accidents  that  could  cause  undue  risk  to  the 
health  and  safety  of  the  public.  This  analysis  determines  the  objectives,  performance  requirements,  and  constraints  of  the 
design,  and  sets  a  framework  for  understanding  the  role  of  controllers  (whether  personnel  or  system)  in  regulating  plant 
processes. 

Function  allocation  is  the  analysis  of  the  requirements  for  plant  control  and  the  assignment  of  control  functions  to  (1) 
personnel  (e.g.,  manual  control),  (2)  system  elements  (e.g.,  automatic  control  and  passive,  self-controlling  phenomena),  and 
(3)  combinations  of  the  two  (e.g.,  shared  control  and  automatic  systems  with  manual  backup).  It  seeks  to  enhance  overall 
plant  safety  and  reliability  by  exploiting  the  strengths  of  personnel  and  system  elements,  including  improvements  that  can 
be  achieved  through  assigning  control  to  these  elements  with  overlapping  and  redundant  responsibilities.  Function 
allocation  should  be  based  upon  HFE  principles  using  a  structured  and  well-documented  methodology  that  provides 
personnel  with  logical,  coherent,  and  meaningful  tasks. 

Element  4  -  Task  Analysis 

Task  analysis  is  the  evaluation  of  the  demands  of  performance  on  personnel  to  identify  the  task  requirements  for 
accomplishing  the  functions  allocated  to  them.  It  defines  the  HSI  requirements  for  supporting  their  accomplishment  of 
tasks.  (It  also  identifies,  by  exclusion,  what  is  not  needed  in  the  HSI).  People  perform  tasks  to  meet  their  fiinctional 
responsibilities.  Although  there  is  no  precise  definition  of  a  task  in  the  abstract,  a  task  is  a  group  of  related  activities  that 
have  a  common  objective  or  goal. 

The  objective  of  this  element  is  to  assure  that  the  applicant’s  task  analysis  identifies  the  requirements  of  the  tasks  that 
personnel  must  perform.  The  task  analysis  should  (1)  provide  one  of  the  bases  for  making  decisions  on  design,  (2)  assure 
that  human  performance  requirements  do  not  exceed  human  capabilities,  (3)  be  used  as  basic  input  for  developing 
procedures,  (4)  be  used  as  basic  information  for  developing  the  staffmg,  training,  and  communication  requirements  of  the 
plant,  and  (5)  form  the  basis  for  specifying  the  requirements  for  the  displays,  data  processing,  and  controls  needed  to  carry 
out  tasks. 
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Element  5  -  Staffing 


Plant  staffing  is  an  important  consideration  throughout  the  design  process.  Initial  staffing  levels  may  be  established  as 
design  goals  early  in  the  process  based  on  experience  with  previous  plants,  customers’  requirements,  initial  analyses,  and 
government  regulations.  However,  the  acceptability  of  the  staffing  goals  and  assumptions  should  be  examined  as  the  design 
of  the  plant  proceeds.  The  objective  of  the  staffing  review  is  to  ensure  that  the  applicant  has  systematically  analyzed  the 
requirements  for  the  number  and  qualifications  of  personnel,  including  a  thorough  understanding  of  the  tasks’  requirements 
and  regulatory  requirements. 

Element  6  -  Human  Reliability  Analysis 

Human  Reliability  Analysis  (HRA)  seeks  to  evaluate  the  potential  for,  and  mechanisms  of,  human  error  that  may  affect 
plant  safety.  Thus,  it  is  an  essential  element  in  achieving  the  HFE  design  goal  of  providing  operator  interfaces  that  will 
minimize  operator’s  errors,  allow  their  detection,  and  provide  recovery  capability.  HRA  has  quantitative  and  qualitative 
aspects,  both  useful  for  HFE  purposes.  The  HRA  should  be  conducted  as  an  integrated  activity  to  support  both  the  HFE  and 
HSI  design  and  probabilistic  risk  assessment  (PRA).  The  PRA  and  HRA  should  be  first  performed  early  in  the  design 
process  to  provide  insights  and  guidance  both  for  systems  design  and  for  HFE  purposes.  The  quality  of  the  HRA  depends, 
in  large  part,  on  the  analyst’s  understanding  of  personnel  tasks,  the  information  related  to  the  tasks,  and  the  factors  which 
influence  human  performance.  Accordingly,  the  HRA  might  be  carried  out  interactively  as  the  design  progresses. 

By  developing  an  understanding  of  the  causes  and  modes  of  human  error,  the  HRA  can  provide  valuable  insights  into  the 
desirable  characteristics  of  the  HSI  design;  consequently,  special  attention  should  be  paid  to  those  scenarios,  critical  human 
actions,  and  HSI  components  that  were  identified  by  HRA  and  PRA  analyses  as  being  important  to  the  plant’s  safety  and 
reliability. 

The  objectives  of  this  review  are  to  ensure  that  (1)  the  applicant  has  addressed  human  error  mechanisms  in  designing  the 
plant’s  HFE;  that  is,  considered  the  HSIs,  procedures,  shift  staffing,  and  training,  to  minimize  the  likelihood  of  personnel 
error,  and  ensure  errors  are  detected  and  recovered  from,  and  (2)  the  HRA  effectively  integrates  the  HFE  program  and  PRA 
and  risk  analysis. 

Element  7  -  Human-System  Interface  Design 

The  selection  of  available  HSIs  and  the  design  of  new  ones  should  result  from  a  process  which  considers  function  and  task 
requirements,  operational  considerations  (e.g.,  the  full-mission  context  within  which  the  HSI  will  be  used),  and  the  crew’s 
personal  safety.  The  HSI  should  be  designed  using  a  structured  methodology  that  should  guide  designers  in  identifying  the 
required  information  and  controls,  in  identifying  and  selecting  candidate  HSI  approaches,  and  in  the  final  design  of  HSIs. 

It  should  cover  the  development  and  use  of  HFE  guidelines  and  standards  that  are  specific  to  the  HSI  design,  and  provide 
guidance  for  resolving  differences  between  different  HFE  guidelines.  It  also  should  address  the  use  of  analysis  and 
evaluation  methodologies  for  dealing  with  design  issues.  The  availability  of  an  HSI  design  methodology  will  help  ensure 
standardization  and  consistency  in  applying  HFE  principles. 

The  objective  of  this  review  element  is  to  evaluate  the  process  by  which  HSI  design  requirements  are  developed  and  HSI 
designs  are  selected  and  refined.  The  review  should  assure  that  the  applicant  has  appropriately  translated  function  and  task 
requirements  to  the  alarms,  displays,  controls,  and  aids  available  to  the  crew.  The  applicant  should  have  systematically 
applied  HFE  principles  and  criteria  (along  with  all  other  function,  system,  and  task  design  requirements)  in  identifying  HSI 
requirements,  selecting  and  designing  HSIs,  and  resolving  HFE  and  HSI  design  problems.  The  process  and  the  rationale  for 


NUREG/CR-6637 


1-6 


1  INTRODUCTION 


the  HSI  design  should  be  documented  for  review  (including  the  results  of  trade-off  studies,  other  analyses  and  evaluations, 
and  the  rationale  for  choosing  design  and  evaluation  tools). 

Element  8  -  Procedure  Development 

Procedures  are  an  essential  component  of  the  HSI  design  and  should  be  developed  from  the  same  design  process  and 
analyses  as  the  other  components  of  the  HSI  (e.g.,  displays,  controls,  operator  aids),  and  evaluated  in  the  same  way.  This 
will  help  to  assure  their  full  integration  in,  and  consistency  with,  the  HSI. 

The  objective  of  this  review  is  to  ensure  that  the  applicant’s  procedure  development  program  will  result  in  procedures  that 
support  and  guide  human  interactions  with  plant  systems,  and  control  plant-related  events.  Human  engineering  principles 
and  criteria  should  be  applied,  along  with  all  other  design  requirements  to  develop  procedures  that  are  technically  accurate, 
comprehensive,  explicit,  easy  to  use,  and  validated. 

Element  9  -  Training  Program  Development 

Training  plant  personnel  is  an  important  factor  in  assuring  safe,  reliable  operation.  The  NRC  requires  a  systems  approach 
to  training  and  also  requires  that  it  is  based  on  the  systematic  analysis  of  Job  and  task  requirements.  The  HFE  analyses 
associated  with  the  HSI  design  process  provide  a  valuable  understanding  of  such  task  requirements.  Therefore,  training 
development  should  be  coordinated  with  the  other  elements  of  the  HFE  design  process. 

The  objective  of  this  review  is  to  ensure  that  the  applicant  establishes  an  approach  for  developing  personnel  training  that 
incorporates  the  elements  of  a  systems  approach,  and 

•  Evaluates  the  knowledge  and  skill  requirements  of  personnel 

•  Coordinates  the  development  of  the  training  program  with  the  other  elements  of  the  HFE  design  process 

•  Implements  the  training  effectively  in  a  manner  consistent  with  human  factors  principles  and  practices. 

Element  10  -  Human  Factors  Verification  and  Validation 

Verification  and  validation  (V&V)  evaluations  seek  to  comprehensively  determine  that  the  final  design  conforms  to  HFE 
design  principles,  and  enables  personnel  to  successfully  and  safely  perform  their  tasks  to  achieve  operational  goals.  This 
review  involves  five  V&V  evaluations,  the  objectives  of  which  are  to  assure  the  following; 

•  HSI  Task  Support  Verification  -  The  HSI  design  provides  the  information,  displays,  and  controls  that  are  necessary 
to  support  the  operators’  tasks,  i.e.,  all  necessary  alarms,  displays,  controls,  and  Job-performance  aids. 

•  HFE  Design  Verification  -  The  manner  in  which  information,  displays,  and  controls  are  presented  in  the  HSI 
design  is  consistent  with  human  cognitive  and  physiological  limitations,  i.e.,  conforms  to  HFE  principles, 
guidelines,  and  standards. 

•  Integrated  System  Validation  -  The  integrated  design  including  all  information,  displays,  and  controls  can  be 
effectively  operated  by  personnel  to  satisfy  all  performance  requirements  associated  with  plant  safety  and 
operational  goals. 
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•  Human  Factors  Issue  Resolution  Verification  -  The  final  HSI  design  resolves  all  identified  HFE  issues  in  the 
tracking  system. 

•  Final  Plant  HFEJHSI  Design  Verification  -  The  final  HSI  design,  as  built  and  installed  (i.e.,  at  a  nuclear  power 
station),  conforms  to  the  verified  and  validated  final  design  resulting  from  the  HFE  design  process. 

Support  for  the  detailed  aspects  of  V&V  activities  is  given  in  NUREG-0700,  Rev.  1. 

1.2.3  Human-System  Interface  Design  Review  Guideline  (NUREG-0700) 

NUREG-0700,  Rev.  1  (O’Hara  et  al.,  1996)  provides  HFE  guidance  to  the  NRC  staff  for  reviewing  HSI  designs.  These 
reviews  may  address  staff  investigations  of  events  involving  human  performance,  voluntary  modifications  made  to  HSIs 
(e.g.,  where  there  is  an  unreviewed  safety  question  involving  the  HSI  or  human  performance),  and  reviews  of  the  HSI  for  a 
new  plant.  The  document  is  divided  into  two  parts.  Part  1,  Review  Methodology  and  Procedures,  describes  aspects  of  the 
HSI  design  review  process  that  are  important  to  identifying  and  resolving  discrepancies  in  human  engineering  that  could 
adversely  affect  plant  safety.  Part  2,  HFE  Guidelines,  contains  guidance  on  different  aspects  of  HSI  technology 
implemented  in  NPPs. 

The  overall  purpose  of  the  HSI  design  review  is  to  ensure  that  it  supports  safe,  efficient,  and  reliable  task  performance.  This 
is  accomplished  by  systematically  identifying  and  resolving  deficiencies  in  design,  called  human  engineering  discrepancies 
(HEDs),  that  could  adversely  affect  plant  safety.  The  scope  of  the  HSIs  included  in  the  review  is  defined  on  the  basis  of  their 
function  and  not  their  physical  location.  Relevant  HSIs  include  all  alarms,  displays,  controls,  job-performance  aids,  and 
workstation  and  workplace  layouts.  Also  included  are  the  environmental  conditions  in  which  the  HSIs  are  operated,  such  as 
lighting,  noise,  temperature,  and  humidity.  The  methodology  includes  the  following  major  phases  of  activity: 

Planning  Phase  -  An  HSI  design  review  plan  should  be  developed  that  adequately  defines  the  evaluation  in  terms  of  review 
goals  and  scope,  review  team,  management  process,  and  technical  approach. 

Preparatory  Analysis  Phase  -  The  analyses  that  make  up  the  preparatory  phase  provide  valuable  information  which  supports 
the  establishment  of  requirements  for  the  HSI  design  process.  These  analyses  also  provide  the  technical  basis  and  criteria 
for  conducting  HSI  design  verifications  and  validation,  i.e.,  they  identify  human  performance  issues  and  task  requirements 
with  which  the  HSI  should  be  evaluated.  A  systems  approach  should  be  used  to  identify  HSI  design  requirements.  Three 
main  analyses  include 

•  Operating  Experience  Review  -  Operating  experience  should  be  reviewed,  including  examining  plant  performance 
reports  and  other  documents,  and  surveying  personnel  to  identify  HSI-related  human  performance  issues. 

•  Function  and  Task  Analysis  -  System  functions  and  personnel  functions  and  tasks  should  be  reviewed  to  identify 
HSI  requirements  and  performance  criteria  for  personnel  tasks. 

•  HSI  Inventory  and  Characterization  -  An  inventory  of  the  HSI  should  be  developed,  also  describing  its 
characteristics,  functions,  and  performance  features. 

HSI  Design  Verifications  and  Validation  Phase  -  HSI  design  verifications  and  validation  should  be  performed  to  ensure  that 
HEDs  have  been  appropriately  identified  and  documented.  Because  no  one  method  is  likely  to  be  sufficiently 
comprehensive,  it  may  be  necessary  to  perform  a  series  of  analyses: 
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•  HSl  Task  Support  Verification  -  This  evaluation  verifies  that  the  HSI  supports  all  identified  personnel  task 
requirements  as  defined  by  task  analyses.  HEDs  are  identified  for  (1)  task  requirements  that  are  not  fully  supported 
by  the  HSI,  and  (2)  the  presence  of  HSI  components  which  may  not  be  needed  to  support  tasks. 

•  HFE  Design  Verification  -  This  evaluation  verifies  that  the  HSI  is  designed  and  implemented  to  account  for 
human  capabilities  and  limitations.  HEDs  are  identified  if  the  design  is  inconsistent  with  HFE  guidelines  in 
NUREG-0700.  Rev.  1. 

•  Integrated  System  Validation  -  This  evaluation  validates  that  the  integrated  HSI  design  enables  the  performance 
requirements  for  safe  operation  to  be  accomplished  without  imposing  excessive  workload.  HEDs  are  identified  if 
task-performance  criteria  are  not  met,  or  if  the  HSI  imposes  a  high  workload  on  personnel. 

HEP  Resolution  Phase  -  This  phase  should  ensure  that  HEDs  have  been  assessed,  and  that  important  ones  have  been 
resolved.  The  assessment  evaluates  the  safety  significance  of  HEDs;  those  having  no  particular  safety  significance  may  be 
analyzed  for  improvement,  but  on  a  lower-priority  basis.  Once  corrective  actions  are  identified,  a  plan  to  implement  them 
should  be  developed,  including  (1)  evaluation  of  the  installation  and  operation  of  all  HSI  modifications,  and  (2)  correction 
of  any  problems  with  implementation. 

The  overall  NUREG-0700  review  process  accommodates  tailoring  based  on  review  needs.  In  Section  1.4.5,  Scope  of  the 
Review  Process,  the  guidance  indicates; 

The  staff  s  review  of  an  applicant’s  HSI  design  review  process  as  described  above  and  in  the  remaining  sections  of  Part  I,  is  a 
comprehensive,  detailed  evaluation.  Under  certain  circumstances,  such  as  a  preliminary  review  of  an  applicant’s  HSI  design 
prototype,  or  a  focused  review  of  one  aspect  of  the  HSI  that  might  be  implicated  in  an  incident  involving  human  performance, 
the  applicant’s  HSI  design  review,  and  correspondingly,  the  focus  of  the  staffs’  review,  could  be  more  limited,  (pp.  14-15) 

Such  an  approach  also  is  suggested  as  part  of  the  detailed  review  process.  For  example,  under  the  planning  phase  criteria, 
the  NUREG  states: 

For  a  major  HSI  upgrade  or  redesign,  the  applicant’s  design  assumptions  or  constraints  should  be  clearly  identified.  An 
assumption  or  constraint  is  an  aspect  of  the  design,  such  as  the  use  of  touch  screens,  that  is  an  input  to  the  HSI  design  process 
rather  than  the  result  of  HFE  analyses.  The  HSI  design  review  should  seek  to  evaluate  the  validity  of  such  assumptions  for  the 
implementing  design  under  review,  (p.  17) 

Thus,  consistent  with  the  SRP  and  NUREG-071 1,  NUREG-0700  reflects  the  use  of  a  graded  approach  to  reviewing  the 
design  process  and  products. 

In  addition  to  NUREGs  0800,  0711,  and  0700,  NRC  inspection  manuals  are  used  in  reviews.  These  are  discussed  in  the 
next  section. 

1.2.4  Nuclear  Regulatory  Commission  Inspection  Manual 

The  NRC  Inspection  Manual  provides  guidance  for  developing  and  conducting  the  NRC  inspection  program.  Chapter 
0030,  Policy  and  Guidance  for  Development  of  NRC  Inspection  Manual  Programs,  states  that  the  objectives  of  the 
inspection  programs  are  to  (1)  provide  a  basis  for  recommending  issuance,  denial,  continuation,  modification,  or  revocation 
of  an  NRC  permit  or  license;  (2)  identify  conditions  within  areas  inspected  that  may  adversely  affect  public  safety  so  that 
appropriate  corrective  action(s)  can  be  taken;  (3)  determine  the  level  of  effectiveness  of  the  licensee’s  performance  in 
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general,  as  well  as  in  each  inspection  program  area;  and  (4)  determine  the  status  of  compliance  with  NRC  regulations, 
licenses,  and  orders.  These  objectives  are  carried  out  through  routine  and  reactive  inspections.  Routine  inspections  are 
systematic,  scheduled  reviews  of  licensee  activities.  Reactive  ones  focus  on  a  particular  aspect  of  the  licensee’s  operation  in 
response  to  a  specific  event  or  condition. 

The  following  describes  chapters  that  may  be  particularly  relevant  to  reviewing  hybrid  HSIs. 

Inspection  Procedure  4 1500:  Training  and  Qualification  Effectiveness 

The  objectives  of  this  inspection  procedure  (NRC,  1995a)  are  to  ensure  that  training  is  an  appropriate  response  to  identified 
problems  in  performance  and  that  the  training  and  qualification  programs  for  NPP  personnel  are  developed,  implemented, 
evaluated,  documented,  and  maintained  as  required  under  10  CFR  50.120  and  allowed  by  10  CFR  55.  This  inspection 
evaluates  the  performance  of  NPP  workers,  the  methods  for  licensee  training  and  qualification  (e.g.,  classroom,  laboratoiy, 
simulation  devices,  on-the-job),  and  the  effectiveness  of  implementing  the  systems  approach  to  training.  The  applicant’s 
training  program  may  be  inspected  as  a  result  of  performance-related  opierational  events. 

Inspection  Procedure  52001:  Digital  Retrofits  Receiving  Prior  Approval,  and  Inspection  Procedure  52002:  Digital  Retrofits 

Not  Receiving  Prior  Approval 

These  two  inspection  procedures  deal  with  the  review  of  digital  retrofits  (modifications)  in  NPPs.  Inspection  Procedure 
52001  (NRC,  1995b)  covers  those  previously  approved  by  the  NRC.  Its  objectives  are  (1)  to  ensure  that  digital  systems 
previously  reviewed  by  the  NRC  staff  are  installed,  operated,  and  maintained  according  to  the  safety  evaluation,  and  in 
accordance  with  the  manufacturer’s  design  and  operating  recommendations  (as  appropriate),  and  licensee’s  commitments, 
and  (2)  to  assess  failures  of  the  digital  systems,  their  modifications  and  maintenance  for  their  effect  on  the  systems’ 
functions,  and  for  potential  generic  concerns. 

Inspection  Procedure  52002  (NRC,  1995c)  focuses  on  digital  retrofits  that  have  not  received  NRC  approval.  Its  objectives 
are  (1)  to  ensure  that  the  licensee  has  properly  considered  the  guidance  for  effective  design  of  the  digital  system  and 
satisfied  the  plant-specific  licensing  basis;  (2)  to  ensure  that  the  licensee  has  properly  addressed  the  regulatory  requirements 
of  10  CFR  50.59  on  unreviewed  safety  questions;  and  (3)  to  assess  failures  of  digital  systems,  their  modifications  and 
maintenance  for  their  effect  on  the  systems’  functions,  and  for  potential  generic  concerns. 

While  the  detailed  criteria  of  Inspection  Procedures  52001  and  52002  differ,  their  general  approaches  are  similar.  Each 
have  criteria  addressing  the  following: 

•  Verification  of  the  design  and  its  documentation  against  design  requirements 

•  Reliability  considerations  (setpoints  and  related  uncertainty  terms) 

•  Procedures  and  practices 

•  Training 

•  Resolution  of  previous  failures  (i.e.,  an  operating-experience  review). 
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1.2.5  Summary 

The  NRC’s  HFE/HSl  review  and  inspection  process  provides  guidance  on  evaluating  the  design,  evaluation,  and 
implementation  processes.  In  applying  it  to  hybrid  HSIs  in  the  context  of  plant  modifications,  the  guidance  is  limited  in  two 
respects.  First,  while  it  allows  tailoring  of  the  review  methods  and  criteria  to  a  unique  individual  review,  there  is  no 
guidance  to  assist  in  identifying  the  process  elements  and  criteria  that  are  needed.  The  extent  of  plant  modifications  can 
range  significantly,  e.g.,  from  a  replacement  “in-kind”  of  a  single  HSI  component  to  an  extensive  modification  of  the 
control  room  from  analog  to  digital  technology.  The  effects  of  the  modifications  can  impact  personnel  by  changing  the 
following: 

•  Personnel  role  -  functions  and  responsibilities  of  personnel,  e.g.,  caused  by  a  change  in  the  plant’s  automation  due 
to  replacing  a  control  system 

•  Primary  tasks  —  specific  tasks  performed  by  personnel  to  accomplish  their  functions  and  responsibilities 

•  Secondary  tasks  -  methods  of  interacting  with  the  HSI  and  the  demands  it  imposes 

•  Personnel  factors  -  required  qualifications  or  training  of  personnel. 

Thus,  guidance  on  tailoring  is  important  and  is  needed. 

The  second  limitation  of  the  guidance  is  that  is  does  not  specifically  address  the  unique  considerations  of  integrating  new 
and  old  HSI  components.  As  discussed  elsewhere  (O’Hara,  Stubler,  and  Higgins,  1996),  unique  considerations  are 
associated  with  modifying  HSIs.  For  example,  there  are  unique  demands  on  crews  during  transitions  from  old  to  new 
systems.  Thus,  special  considerations  for  each  element  of  the  design  process  review  need  to  be  identified,  and  the  required 
guidance  developed. 

It  is  important  in  the  present  effort  to  consider  these  limitations,  and  to  ensure  that  the  guidance  developed  is  fully 
consistent  and  integrated. 

1.3  Earlier  Research  on  HSI  Upgrades 

A  review  of  the  HFE  literature  and  industrial  experience  on  introducing  advanced  technologies  into  traditional  HSIs  found 
that  in  such  situations  special  challenges  can  be  posed  to  personnel  performance  (O’Hara,  Stubler,  and  Higgins,  1996). 
These  concerns  were  further  refined  in  the  characterizations  of  this  issue  by  Stubler  et  al.  (1996)  and  Stubler  and  O’Hara 
(1996a).  Some  identified  concerns  included  compatibility  of  new  technology  with  the  existing  HSI  technology,  and  new 
demands  imposed  by  new  technology  (e.g.,  changes  in  the  operator’s  roles  due  to  changes  in  automation),  human 
performance  considerations  associated  with  developing  and  conducting  personnel  training,  and  those  associated  with  the 
processes  by  which  upgrades  are  installed  and  put  into  service.  Stubler  et  al.  (1996)  describe  this  topic  and  analyze  its 
potential  safety  significance. 

1.4  Organization  of  the  Report 

This  report  is  divided  into  two  parts.  Part  1  describes  the  methodology  for  developing  guidance  and  its  technical  basis.  The 
objectives  of  the  study  are  described  in  Section  2,  and  the  development  methodology  is  described  in  Section  3.  Sections  4 
and  5  present  the  technical  basis  for  establishing  of  HFE  guidance.  Section  4  discusses  the  information  processing 
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capabilities  of  humans  and  how  performance  may  be  affected  by  changes  in  plant  systems  or  the  HSI.  Section  5  continues 
the  technical  basis  with  a  discussion  of  how  these  human  performance  issues  should  be  considered  during  the  design, 
development,  and  implementation  of  an  upgrade.  This  discussion  is  organized  around  the  10  review  elements  of  NUREG- 
0711.  It  describes  how  the  general  review  criteria  in  NUREG-071 1  should  be  applied  to  a  specific  upgrade.  The  actual  use 
of  the  technical  information  for  developing  guidance  is  discussed  in  Section  6.  The  guidance  development  effort  is 
summarized  in  Section  7.  Full  references  to  the  literature  cited  are  listed  in  Section  8. 

Part  2  of  the  document  contains  results  of  the  guidance  development  process;  guidelines  are  organized  according  to  the  ten 
review  elements  of  NUREG-071 1. 
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2  OBJECTIVE 


The  objective  of  this  study  was  to  develop  HFE  review  guidance  to  address  design,  analysis,  evaluation,  and  implementation 
of  hybrid  HSls  based  on  a  technically  valid  methodology.  To  support  this  objective,  the  following  tasks  were  performed: 

•  Development  of  a  technical  basis  for  guidance  based  on  human  performance  research  and  accepted  HFE 
procedures  and  practices  for  system  development 

•  Development  of  HFE  review  guidelines  in  a  format  that  is  consistent  with  NUREG-07 1 1 

•  Identification  of  remaining  issues  for  which  human  performance  research  and  accepted  HFE  procedures  and 
practices  were  insufficient  to  support  the  development  of  NRC  review  guidance 
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3.1  Overview 


The  overall  methodology  for  guidance  development  for  NUREG-0700  is  shown  in  Figure  3. 1 ;  the  process  is  discussed  in 
detail  elsewhere  (O’Hara,  Brown,  and  Nasta,  1996;  Stubler  and  O’Hara,  1996a).  The  portion  of  the  methodology  applicable 
to  this  project  is  boxed  in  the  figure.  This  section  of  the  report  describes  the  general  rationale  behind  developing  guidance. 


Figure  3.1  Major  Steps  in  the  Development  of  NUREG-0700  Guidance 


The  methodology  for  guidance  development  was  directed  by  the  following  objectives: 

•  Establish  a  process  that  will  result  in  valid,  technically  defensible,  review  criteria. 

•  Establish  a  generalizable  process  that  can  be  applied  to  any  aspect  of  HSI  technology  for  which  review 

guidance  is  needed. 

•  Establish  a  process  that  optimally  uses  available  resources;  i.e.,  a  cost-effective  methodology. 

The  methodology  places  a  high  priority  on  establishing  the  validity  of  the  guidelines,  which  is  defined  along  two 
dimensions:  internal  and  external  validity.  Internal  validity  is  the  degree  to  which  the  individual  guidelines  are  based  on 
an  auditable  technical  basis.  The  technical  basis  is  the  information  upon  which  the  guideline  is  established  and  justified. 
Some  guidelines  may  be  based  on  technical  conclusions  from  a  preponderance  of  empirical  research  evidence,  some  on  a 
consensus  of  existing  standards,  while  others  are  based  on  judgement  that  a  guideline  represents  good  practices. 

Maintaining  an  audit  trail  from  each  guideline  to  its  technical  basis  serves  several  purposes  by  enabling 

•  the  technical  merit  of  the  guideline  to  be  evaluated  by  others, 

•  a  more  informed  application  of  the  guideline,  since  its  basis  is  available  to  users,  and 

•  deviations  or  exceptions  to  the  guideline  to  be  evaluated. 

External  validity  is  the  degree  to  which  the  guidelines  are  subjected  to  independent  peer  review.  The  peer  review  process  is 
a  good  method  of  screening  guidelines  for  conformance  to  accepted  HFE  practices,  and  for  comparing  guidelines  to  the 
practical  operational  experience  of  HSIs  in  real  systems. 
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For  individual  guidelines,  these  forms  of  validity  can  be  inherited  from  the  documents  that  form  their  technical  basis.  Some 
HFE  standards  and  guidance  documents,  for  example,  already  have  good  internal  and  external  validity.  If  validity  is  not 
inherited,  however,  it  should  be  established  as  part  of  the  guidance-development  process. 

Figure  3.2  depicts  the  process  used  to  develop  the  technical  basis  and  guidance.  It  emphasizes  those  information  sources 
that  have  the  highest  degree  of  internal  and  external  validity.  Thus,  primary  and  secondary  source  documents  were  sought 
as  sources  of  guidance  first,  followed  by  tertiary  source  documents,  basic  literature,  industry  experience,  and  other  sources. 
From  these  fundamental  data,  design  principles  and  lessons  from  industry  experience  were  identified,  and  the  guidance  was 
developed.  For  specific  aspects  of  the  topic,  in  which  there  was  an  inadequate  technical  basis,  unresolved  research  issues 
were  defined.  Thus,  the  analysis  of  information  led  to  the  development  of  both  guidance  and  issues.  The  resulting  guidance 
documentation  includes  HFE  guidelines,  technical  basis,  the  development  methodology,  and  unresolved  research  issues. 

Each  of  the  steps  of  this  research  -  topic  characterization,  technical-basis  development,  guidance  development  and 
documentation,  issue  identification,  and  peer  review  -  is  discussed  in  greater  detail  in  the  following  sections. 

3.2  Characterization 

The  first  step  in  developing  guidance  for  an  HSI  technology  is  to  identify  specific  design  characteristics  for  which  guidance 
is  needed.  This  characterization  provides  a  framework  for  organizing  guidelines  and  directing  HFE  reviews.  However,  the 
current  review  topic,  HSI  and  Plant  Modernization  Process,  is  directed  toward  a  design  process,  rather  than  specific  design 
features.  An  acceptable  methodology  for  reviewing  an  HSI  design  process  is  given  in  Chapter  18,  Human  Factors 
Engineering,  of  the  SRP  (NUREG-0800)  and  in  NUREG-071 1.  The  review  elements  of  this  methodology  were  described  in 
Section  1 .2.2.  The  topical  organization  of  these  documents  provides  a  framework  with  which  to  organize  the  guidance 
developed  in  this  document.  Therefore,  it  was  not  necessary  to  develop  a  separate  characterization. 

Because  the  NUREG-071 1  organization  was  adopted  as  the  structure  for  this  guidance,  each  section  of  the  developed 
guidance  relates  to  one  of  its  ten  review  elements,  and  the  present  guidance  complements  the  NUREG-07 1 1  guidance. 

3.3  Development  of  the  Technical  Basis 

The  technical  basis  for  this  topic  was  developed  according  to  the  process  described  in  Section  3.1  and  depicted  in  Figure 
3.2.  First,  primary  source  documents  were  sought  to  provide  a  valuable  starting  place.  These  were  HFE  standards  and 
guidance  possessing  internal  and  external  validity;  that  is,  the  documents  generally  had  their  own  research  bases.  The 
developers  of  these  documents  considered  research  and  operational  experience  and,  using  their  knowledge  and  expertise, 
developed  HFE  guidelines.  These  primary  source  documents  were  often  extensively  peer  reviewed.  They  provided 
tremendous  value-added  to  individual  research  reports.  They  were  developed  by  experts  who  consider  the  applicability  and 
generalizability  of  research  to  real  systems,  include  their  knowledge  and  expertise  gained  through  operational  experience 
and  applying  guidance,  and  modify  the  guidance  based  on  extensive  peer  review.  Included  in  this  category  were  HFE 
review  guidance  documents  that  are  currently  used  in  reviewing  NPP’s  HSIs,  such  as  NUREG-0800  (SRP),  NUREG-071 1, 
and  NUREG-0700. 

The  development  of  the  technical  basis  also  considered  the  other  sources  identified  in  Figure  3.2,  except  original  research. 
Secondary  sources  were  documents  for  which  either  internal  or  external  validity  had  been  established.  Documents  for 
which  neither  was  established  were  considered  tertiary  sources.  The  preference  was  to  use  documents  for  which  validity 
was  established. 
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Figure  3.2  Technical  Basis  and  Guidance  Development  Process 
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In  addition  to  these  sources,  the  results  from  basic  literature  were  analyzed  (articles  from  technical  journals,  reports  from 
research  organizations,  and  papers  from  technical  conferences).  When  guidance  was  based  on  basic  literature,  engineering 
Judgement  was  required  to  generalize  from  the  unique  aspects  of  individual  experiments  to  actual  applications  in  the 
workplace.  While  research  information  is  a  valuable  part  of  the  guidance-development  process,  it  usually  cannot  be  adopted 
blindly;  the  results  must  be  interpreted  in  the  context  of  real-world  tasks  and  systems  using  Judgement  based  on  professional 
and  operational  experience. 

Industry  experience  was  also  used,  such  as  published  case  studies,  surveys  and  interviews  with  knowledgeable  domain 
experts.  Included  in  this  category  were  the  findings  from  visits  to  a  variety  of  control  centers  that  were  made  in  conjunction 
with  this  (O’Hara,  Stubler,  and  Higgins,  1996)  and  other  research  projects  (e.g.,  Heslinga  and  Herbert,  1993).  Although 
such  information  lacks  a  rigorous  experimental  basis  (and  thus  a  measure  of  validity),  it  is  highly  relevant. 

The  technical-basis  development  methodology  stopped  at  this  point.  Where  additional  issues  are  identified,  original 
research  can  be  undertaken.  This  approach  has  the  advantage  of  focusing  on  specific  issues  of  interest,  and  has  both  high 
relevance  and  a  sound  experimental  basis  from  which  to  establish  validity.  It  is  generally  given  the  lowest  priority  because 
of  the  time  and  cost  required  to  conduct  original  research. 

The  technical  basis  was  organized  into  two  sections.  Section  4  discusses  the  general  characteristics  of  human  performance 
in  complex  human-machine  systems  that  may  be  affected  by  changes  to  the  HSI.  It  is  based  on  concepts  from  an  accepted 
body  of  literature  on  human  information  processing  and  is  augmented  with  research  data  that  address  specific  human- 
machine  systems.  Section  5  discusses  HFE  considerations  within  the  context  of  the  design,  evaluation,  and  implementation 
process.  This  section  presents  HFE  considerations  identified  from  research  studies,  reviews  of  industry  experiences  and 
practices,  and  handbooks.  It  also  draws  upon  the  high-level  guidance  in  NUREG-0800,  NUREG-071 1,  and  NUREG-0700. 

3.4  Development  and  Documentation  of  Guidance 

Once  the  technical  information  was  assembled,  a  draft  set  of  guidelines  was  developed  from  it.  The  guidelines  were 
organized  and  specified  in  a  standard  format  (discussed  in  Section  6).  The  guidelines  themselves  are  given  in  Section  9  of 
Part  2  of  this  document. 

3.5  Identification  of  the  Issues 

Where  there  was  insufficient  information  for  a  technical  basis  upon  which  to  develop  valid  design  review  guidance,  an  issue 
was  defined.  These  issues  are  described  in  Section  5.6.  From  a  research  standpoint,  they  reflect  aspects  of  the  design, 
evaluation,  or  implementation  of  NPP  upgrades  whose  resolution  will  require  additional  investigation.  From  a  design 
review  standpoint,  these  issues  reflect  HFE  considerations  that  currently  must  be  addressed  on  a  case-by-case  basis.  For 
example,  an  issue  can  be  addressed  as  part  of  design-specific  tests  and  evaluations. 

3.6  Peer  Review 


The  resulting  document  containing  the  technical  basis  and  guidance  was  submitted  for  review  by  individuals  with  topical 
knowledge  and  expertise  including  the  NRC’s  staff  with  expertise  in  human  factors  engineering  and  other  engineering 
fields.  Human  factors  specialists  external  to  the  NRC  who  have  expertise  in  human  performance  in  complex  systems,  such 
as  NPPs  and  aviation,  also  reviewed  the  document.  These  external  reviews  included  evaluations  of  the  topic’s 
characterization  along  the  following  criteria:  clarity,  accuracy,  and  completeness.  The  evaluation  of  the  technical  basis 
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encompassed  the  following  criteria:  organization,  necessity,  sufficiency,  resolution,  and  basis.  Comments  from  the  peer 
reviews  were  incorporated  into  this  document. 
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4  TECHNICAL  BASIS  DEVELOPMENT:  HUMAN  PERFORMANCE 
IN  COMPLEX  HUMAN-MACHINE  SYSTEMS 


To  perform  successfully  in  a  work  environment,  such  as  a  NPP,  personnel  must  develop  the  necessary  knowledge  and  skills. 
When  a  plant  system  or  HSI  is  upgraded,  new  tasks  may  be  created  and  existing  tasks  may  change.  Personnel  must  adapt 
existing  knowledge  and  skills  or  acquire  new  ones,  subject  to  the  capabilities  and  limitations  of  human  information 
processing.  The  following  discusses  those  human  characteristics  (knowledge,  skills,  capabilities,  and  limitations)  that  are 
relevant  to  performance  in  complex  systems,  and  which  affect  the  ability  of  people  to  adapt  to  changes  resulting  from 
upgrades.  This  section  begins  with  a  discussion  of  the  relationship  between  personnel  performance  and  the  HSI  (e.g.,  how 
human  characteristics  affect  the  performance  of  the  plant,  and  vice  versa).  This  is  followed  by  an  outline  of  primary  and 
secondary  tasks  performed  by  personnel.  Next,  the  basic  properties  of  human  information  processing  and  skilled 
performance  that  are  important  for  adapting  to  changes  in  plant  systems  or  the  HSI  are  described.  Finally,  we  discuss 
generic  cognitive  tasks  undertaken  by  operators  in  NPPs  and  their  relationship  to  the  characteristics  of  the  HSI.  This 
section  forms  the  basis  for  Section  5,  which  discusses  human  performance  in  the  context  of  specific  stages  of  the  design  and 
implementation  of  upgrades. 

4.1  The  Effects  of  HSI  Design  on  Personnel  Performance 

The  operator’s  role  in  a  NPP  is  that  of  a  supervisory  controller  (O’Hara,  1994;  O’Hara,  Stubler,  and  Nasta,  1997).  The 
plant’s  performance  is  achieved  through  the  combination  of  the  control  actions  of  human  operators  and  automatic  control 
systems  that  are  under  the  operator’s  supervision.  The  supervisory  role  requires  the  operator  to  monitor  the  behavior  of 
systems  and  automatic  control  systems,  and  intervene,  as  necessary,  to  attain  operational  and  safety  goals.  The  HSI  is  a 
primary  point  of  interaction  between  the  operator  and  the  plant.  It  is  a  major  source  of  information  and  provides  the  media 
through  which  the  operator  acts  on  the  plant. 

The  operator’s  interaction  with  the  plant  may  be  modeled  as  a  causal  chain  (Figure  4.1),  as  described  in  O’Hara  (1997)  and 
O’Hara,  Stubler  and  Nasta  (1997).  When  considering  the  operator’s  effect  upon  the  plant,  this  chain  may  be  viewed  as 
starting  from  the  operator’s  physiological  and  cognitive  processes,  and  proceeding  to  task  performance,  which  manipulates 
the  HSI.  These  manipulations  affect  the  plant’s  components,  systems,  and  ultimately,  its  frmctions.  Those  of  particular 
importance  are  the  critical  safety  functions  (CSFs),  which  are  necessary  for  maintaining  the  plant  in  a  safe  condition. 

When  considering  the  effects  of  upgrades  on  personnel  performance,  this  causal  chain  may  be  viewed  in  the  opposite 
direction.  The  design  of  systems  and  components  affects  the  behavior  of  the  plant.  The  status  of  the  plant’s  functions, 
systems,  and  components  is  communicated  to  the  operator  via  user  interfaces  of  the  HSI.  This  information  imposes 
demands  on  the  cognitive  resources  and  physical  capabilities  of  personnel  in  their  roles  as  supervisory  controllers.  In 
addition,  the  characteristics  of  the  HSI,  including  the  way  information  is  presented  and  control  actions  are  executed,  impose 
additional  demands  on  operators. 

Changes  made  to  a  plant’s  systems  and  components,  as  part  of  an  upgrade,  may  change  the  behavior  of  the  affected  systems. 
For  example,  a  newly  insulled  digital  control  system  may  introduce  new  functions  and  new  control  modes  (e.g.,  higher 
levels  of  automatic  control),  or  cause  a  system  to  respond  differently  to  control  inputs  (e.g.,  act  more  quickly  and  exhibit  less 
drift).  As  an  example  of  new  functions,  some  digital  feedwater  system  upgrades  automatically  switch-over  function  from 
the  auxiliary  to  the  main  feedwater  systems  during  startup;  these  changes  impose  new  demands  on  personnel  as  supervisory 
controllers. 

Changes  made  to  the  HSI  affect  the  way  that  information  is  presented  to  personnel.  For  example,  rather  than  being 
presented  via  individual,  spatially  dedicated  instruments,  the  information  may  be  presented  via  computer-based  video 
display  units  (VDUs).  The  HSI  modifications  may  also  change  the  controls  used  by  the  operator.  For  example,  physical 
control  devices  may  be  replaced  by  soft  controls  (Stubler,  O’Hara,  and  Kramer,  2000),  changing  both  the  appearance  and 
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Figure  4.1  Causal  Links  Between  Human  and  Plant  Characteristics  That  Affect  System  Performance 

behavior  of  the  device.  In  addition,  new  HSI  components  may  include  higher  levels  of  automation.  For  example,  display 
systems  may  include  automation  for  selecting,  processing,  and  presenting  information.  HSI  modifications  may  place  new 
cognitive  and  physiological  demands  on  personnel.  New  cognitive  demands  may  stem  from  the  need  to  understand  which 
information  and  controls  are  being  presented,  and  how  to  access  and  use  them.  Physiological  demands  may  stem  from 
limitations  on  human  perception,  response  time,  and  response  accuracy  associated  with  using  the  new  interfaces. 

4.2  Primary  and  Secondary  Tasks 

The  role  of  an  operator  can  be  conceptualized  as  involving  primary  and  secondary  tasks.  Primary  tasks  are  those  the 
operator  performs  as  part  of  their  functional  role  of  supervising  the  plant.  They  require  a  knowledge  of  the  plant  and 
cognitive  skills  for  making  decisions  and  taking  actions  within  the  limitations  of  human  information  processing 
capabilities.  For  example,  operators  may  be  required  to  monitor  automatic  systems  and  intervene  if  the  systems  fail  to 
perform  acceptably.  The  primary  tasks  involve  the  generic  cognitive  tasks  (i.e.,  monitoring  and  detection,  situation 
awareness,  response  planning  and  implementation)  which  are  described  in  Section  4.4.  Secondary  tasks  are  those  that 
operators  must  perform  when  interacting  with  the  HSI  but  which  are  not  directly  related  to  a  primary  task  (O’Hara,  Stubler, 
and  Nasta,  1997).  They  include  navigating  through  an  information  system,  manipulating  windows  on  a  VDU,  and 
manipulating  job-performance  aids.  Secondary  tasks  result  from  the  design  characteristics  of  the  HSI  (i.e.,  some  designs 
impose  more  secondary  tasks  than  others).  Secondary  tasks  require  knowledge  of  the  HSI  and  specific  skills  for  accessing 


NUREG/CR-6637 


4-2 


4  DEVELOPMENT  OF  THE  TECHNICAL  BASIS:  HUMAN  PERFORMANCE 


and  using  the  information  and  control  features.  Thus,  a  goal  of  HFE  in  HSI  design  should  be  to  minimize  any  negative 
effects  of  the  secondary  tasks  on  personnel  performance. 

Section  4.3  describes  human  information  processing  limits,  capabilities,  knowledge,  and  skills  that  affect  task  performance, 
including  primary  and  secondary  tasks. 


4.3  Basic  Properties  of  Human  Information  Processing  and  Skilled  Performance 

Researchers  have  proposed  various  models  of  human  information  processing.  The  following  describes  some  concepts 
common  to  many  models.  Included  are  discussions  of  some  basic  properties  (capabilities  and  limitations  of  memory  and 
attention)  and  cognitive  skills  (Schlager  et  al.,  1989),  followed  by  a  discussion  of  human  error.  These  concepts  provide  the 
basis  for  the  exploration  of  generic  cognitive  tasks  in  Section  4.4,  and  are  addressed  in  the  personnel-performance  topics 
presented  in  Section  5.  Human  information  processing  is  described  in  more  detail  in  Section  2.3  of  Volume  1  of 
NUREG/CR-5908  (O’Hara,  1994). 

4.3.1  Memory  and  Attention 

A  fundamental  assumption  of  most  models  of  human  information  processing  is  the  existence  of  multiple  types  of  memory 
(e.g.,  Waugh  and  Norman,  1965).  Long-term  memory  is  considered  the  repository  of  knowledge  (i.e.,  things  that  have  been 
learned).  Its  capacity  for  storing  information  is  large  -  considered  by  some  to  be  virtually  unlimited.  A  distinction  is 
commonly  made  between  two  types  of  knowledge:  factual  and  procedural.  Factual  knowledge  is  stored  as  statements  of 
fact,  such  as  “A  reactor  coolant  pump  pumps  coolant  into  the  reactor  vessel  of  a  pressurized  water  reactor.”  This  kind  of 
knowledge  can  usually  be  stated  fairly  easily.  Procedural  knowledge  is  knowledge  of  how  to  perform  actions,  such  as  using 
a  control  to  start  operating  a  particular  piece  of  equipment  or  access  a  specific  page  from  a  display.  Procedural  knowledge 
is  not  easy  to  communicate  through  statements  and  often  requires  demonstrations. 

A  second  type  of  memory  is  working  memory,  sometimes  referred  to  as  active  memory.  It  contains  things  that  are  currently 
in  consciousness  (i.e.,  things  that  one  is  considering  at  the  moment).  Working  memory  may  be  compared  to  a  short  list. 
Unlike  long-term  memory,  the  number  of  items  that  can  be  stored  at  any  time  in  working  memory  is  severely  limited;  some 
say  that  the  limit  is  about  seven  items  (e.g.,  Miller,  1956).  In  attempting  to  exceed  the  limits  of  working  memory,  some 
items  become  lost. 

Although  there  appear  to  be  severe  limits  on  the  number  of  items,  the  size  of  an  item  can  vary.  Thus,  the  amount  of 
information  held  in  working  memory  can  be  increased  by  increasing  the  size  of  the  items  by  constructing  a  meaningful 
group  (“chunk”)  of  information,  associated  with  a  single  label.  This  chunk  then  becomes  a  single  item  to  be  stored  in 
working  memory.  Working  memory  can  hold  a  large  amount  of  information,  despite  its  limitations,  by  storing  the  labels  of 
large,  meaningful  chunks  of  knowledge  (Newell  and  Simon,  1972;  cited  in  Schlager  et  al.,  1989). 

Another  property  of  working  memory  is  that  it  is  the  location  where  conscious  activities  are  carried  out,  such  as  problem 
solving  and  decision  making.  This  is  accomplished  by  combining  information  from  the  environment  and  long-term 
memory  with  intentions  (task  goals).  The  gathering  and  processing  of  this  information  is  mediated  by  the  mechanism  of 
attention,  which  determines  which  items  of  information  will  enter  the  working  memory.  Two  types  of  attention,  selective 
and  sustained,  are  important  to  personnel  performance  in  NPPs. 
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Selective  Attention 


Selective  attention  (e.g.,  Parasuraman  and  Bowers,  1987;  cited  in  Schlager  et  al.,  1989)  is  the  ability  to  shift  attention  to 
important  things  and  ignore  irrelevant  things.  A  common  effect  of  selective  attention  is  popularly  described  as  the  “cocktail 
party  phenomenon,”  in  which  a  person  at  a  crowded  party  is  deeply  involved  in  a  conversation,  and  apparently  ignoring  the 
other  conversations  and  sounds.  Suddenly,  the  person’s  attention  is  drawn  to  another  conversation  when  his  or  her  name  is 
mentioned,  despite  the  fact  that  the  person  was  not  “listening”  to  the  other  conversation.  The  explanation  is  that  people  can 
unconsciously  attend  to  many  things,  but  they  consciously  attend  to  perhaps  only  one  thing  at  a  time.  Certain  stimuli,  such 
as  the  mention  of  one’s  name,  can  shift  in  focus  of  attention. 

Selective  attention  is  driven  by  both  top-down  and  bottom-up  processes  (Wickens  and  Carswell,  1997).  O’Hara  (1994) 
gives  a  general  discussion  of  the  application  of  top-down  and  bottom-up  processing  in  NPP  control  rooms  (CRs).  Bottom- 
up  processes  are  based  on  characteristics  of  the  stimuli  that  affect  salience,  such  as  size,  loudness,  and  brightness.  Bottom- 
up  processing  results  in  attention  to  stimuli  that  have  highly  salient  characteristics.  Top-down  processes  are  developed  from 
past  experiences  and  are  based  on  one’s  understanding  of  the  environment  (i.e.,  mental  model)  and  expectations.  Top-down 
processing  results  in  attention  to  stimuli  that  have  special  significance,  such  as  one’s  name  as  in  the  example  above,  or  to 
information  sources  that  are  likely  to  produce  important  information. 

Sustained  Attention 


Sustained  attention  (e.g.,  Parasuraman  and  Bowers,  1987;  cited  in  Schlager  et  al.,  1989),  also  called  focused  attention 
(Wickens  and  Carswell,  1997),  is  the  ability  to  attend  to  a  single  source  for  an  extended  period.  This  type  of  attention  has 
been  studied  in  signal  detection  tasks  in  which  an  operator  attempts  to  detect  an  target  signal  from  a  background  of  noise 
stimuli. 


4.3.2  Cognitive  Skills  for  Overcoming  Limitations  of  Memory  and  Attention 

The  two  distinctions  described  earlier,  factual  versus  procedural  knowledge  and  long-term  versus  working  memory,  along 
with  attention,  establish  many  of  the  constraints  on  the  mental  activities  that  accompany  skilled  performance.  Because 
conscious  attention  can  only  be  given  to  a  limited  number  of  sets  of  items  at  one  time,  personnel  must  develop  cognitive 
skills  to  handle  the  large  volume  of  changing  information  that  occurs  in  complex  human-machine  systems,  such  as  NPPs. 

The  following  are  four  generic  learning  mechanisms  that  support  the  transition  from  novice  to  experienced  user  in  a  task 
domain  (Schlager  et  al.,  1989).  They  represent  different  means  of  coping  with  the  demands  imposed  on  users  when  new 
technologies  are  introduced  into  their  work  environment.  Applying  these  learning  mechanisms  within  a  task  domain 
results  in  the  acquisition  of  specific  skills  that  allow  the  user  to  attain  a  high  level  of  proficiency. 

Chunking  of  Information 

As  described  above,  people  can  hold  only  a  limited  amount  of  information  in  working  memory  at  one  time.  Consequently, 
when  the  demands  on  working  memory  are  high,  human  performance  may  suffer.  One  mark  of  expertise  is  the  ability  to 
handle  large  quantities  of  information  by  forming  it  into  chunks  relevant  to  the  tasks  of  one’s  particular  domain.  This 
strategy  allows  the  user  to  attend  to  important  information  while  staying  within  the  limits  of  working  memory.  For 
example,  plant  information  presented  to  NPP  operators  becomes  easier  to  remember  and  process  when  groups  of  facts  are 
stored  together  as  meaningful  chunks.  Indications  that  correspond  to  a  single  fault  (e.g.,  loss  of  emergency  feedwater)  or  a 
single  plant  state  (e.g.,  startup)  may  be  organized  in  working  memory  as  a  single  chunk,  rather  than  as  separate  indications. 
Chunking  also  frees  up  some  of  the  limited  resources  of  working  memory,  which  can  be  used  for  attending  to  other 
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indications,  or  for  planning  and  decision  making.  Chunking  may  be  based  on  a  number  of  attributes  of  the  information, 
such  as  causal  relationships  (e.g.,  indications  A,  B,  and  C  are  all  encoded  as  “symptoms  of  condition  X”)  and  structural 
relationships  (e.g.,  components  A,  B,  and  C  are  coded  as  “parts  of  pump  X”). 

Automaticitv  of  Actions 

Automaticity  refers  to  the  ability  to  perform  specific  sets  of  actions  without  committing  extensive  cognitive  resources 
(Mumaw  and  Gabrys,  1996;  Schlager  et  al.,  1989).  These  sets  of  actions  are  also  called  automatic  actions  or  automatic 
procedures  (Schiffrin  and  Schneider,  1977;  cited  in  Schlager  et  al.,  1989).  When  a  user  first  learns  a  new  set  of  actions, 
such  as  how  to  operate  a  particular  control  device,  each  step  of  the  set  requires  attention  and  must  be  executed  separately 
and  consciously  by  the  user.  This  limited  level  of  performance  uses  working  memory  extensively.  It  requires  operators  to 
expend  most  of  their  limited  cognitive  resources  on  performing  relatively  low-level  aspects  of  their  jobs  (e.g.,  remembering 
methods  for  searching,  retrieving,  and  using  displays  and  controls),  thus  preventing  them  from  attending  to  more  critical 
tasks,  such  as  evaluating  plant  conditions,  planning  responses,  and  making  decisions  (Schlager  et  al.,  1989).  To  become  a 
skilled  operator,  one  must  learn  to  “automate”  these  sets  of  actions.  A  familiar  example  is  learning  to  type  on  a  keyboard, 
initially,  this  task  requires  a  great  deal  of  cognitive  resources  in  searching  for  characters  on  the  keyboard  or  trying  to  recall 
which  keys  correspond  to  which  fingers  when  touch-typing.  However,  a  skilled  typist  can  type  quickly  and  accurately, 
relying  little  on  cognitive  resources.  This  freed-up  capacity  may  be  used  instead  for  deciphering  and  comprehending  a 
complex,  hand-written  document. 

Not  all  actions  can  be  performed  “automatically.”  While  it  is  desirable  to  “automate”  low-level  sets  of  actions,  such  as  steps 
for  operating  a  control,  other  actions  require  conscious  control  of  cognitive  resources,  such  as  those  involving  decisions  on 
whether  to  operate  a  control  or  which  value  to  enter.  As  another  example,  some  actions  require  a  great  deal  of  attention, 
such  as  those  requiring  careful  positioning.  Other  sets  of  actions  cannot  be  “automated”  because  some  actions  are  not  the 
same  every  time  they  are  performed.  For  example,  some  actions  in  the  set  required  for  operating  a  control  may  be  unique, 
such  as  determining  a  particular  value  to  enter  as  a  control  setpoint.  In  addition,  performing  actions  without  extensively 
focusing  attention  on  the  task  can  contribute  to  errors  of  execution  (Norman,  1988,  1983,  1981;  Stubler,  O’Hara,  and 
Kramer,  2000). 

Use  of  Selective  Attention 


Performers  learn  through  experience  where  and  when  to  look  in  their  work  environment  to  gain  the  greatest  information, 
and  selectively  focus  attention  on  these  sources.  In  dynamic  environments,  such  as  NPPs,  there  is  a  tendency  for  operators 
to  attend  to  those  sources  that  change  most  frequently  (i.e.,  contain  the  most  information  in  terms  of  bits  per  unit  of  time), 
or  are  likely  to  change  given  the  current  situation.  These  are  examples  of  top-down  processing  (e.g.,  based  on  their 
understanding  of  the  current  situation,  operators  develop  expectations  of  information  sources  that  will  provide  the  most 
useful  information).  Monitoring  based  on  top-down  processing  is  referred  to  as  model-driven  or  knowledge-driven 
monitoring.  Proficient  NPP  operators  rely  on  this  capability  to  allow  them  to  shift  their  attention  between  the  many  sources 
of  information  in  the  CR,  especially  when  situations  change;  they  learn  which  stimuli  represent  things  that  are  likely  to 
require  their  attention. 

When  scanning  large  volumes  of  data,  personnel  also  use  the  bottom-up  processes  by  focusing  on  information  that  is  highly 
salient.  This  provides  one  mechanism  for  prioritizing  information  sources  (i.e.,  highly  salient  information  is  anended  to 
before  other  less-salient  information).  Monitoring  that  is  based  on  bottom-up  processes  is  referred  to  as  data-driven 
monitoring.  (Section  4.4. 1  describes  the  application  of  bottom-up  and  top-down  processes  to  monitoring  tasks). 
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Use  of  Sustained  Attention 


Proficient  performers  develop  the  ability  to  maintain  attention  on  important  information  sources  and  detect  meaningful 
signals  amid  random  fluctuations  with  little  operational  significance.  Computer-based  automation  is  sometimes  introduced 
as  a  way  of  improving  an  operator’s  performance  in  these  detection  tasks.  Rather  than  eliminating  the  human  monitor,  this 
typically  combines  human  and  computer  monitoring.  In  some  cases,  the  combined  monitoring  performance  can  be  worse 
than  that  of  either  one  alone  (Corcoran,  et  al.,  1972;  cited  in  Schlager  et  al.,  1989).  This  case  is  especially  true  when  the 
automation’s  performance  increases  to  the  point  where  the  human  becomes  overly  reliant  upon  it. 

4.3.3  Cognitive  Skills  for  Problem  Solving 

In  addition  to  skills  developed  to  overcome  human  limitations  of  memory  and  attention,  personnel  develop  cognitive  skills 
for  problem  solving.  These  skills  are  closely  linked  to  knowledge  of  the  specific  work  domain  (e.g.,  NPP  operations).  Low- 
level  skills  support  the  execution  of  the  high-level  skills.  Examples  include  operating  a  control  or  selecting  a  display  from 
the  display  systems.  These  skills  are  often  performed  under  conditions  that  allow  them  to  be  trained  for  automaticity. 
High-level  skills  are  related  to  decision  making,  and  are  often  performed  under  variable  conditions,  which  do  not  allow  such 
training  (Schlager  et  al.,  1989). 

Three  high-level  skills  are  particularly  relevant  to  NPP  operations:  planning,  use  of  strategic  knowledge,  and  use  of  mental 
models  (Schlager  et  al,  1989).  Planning  skills  are  involved  with  setting  goals  and  determining  paths  to  them.  Strategic 
knowledge  refers  to  the  use  of  knowledge  of  possible  strategies  (paths)  for  satisfying  a  particular  goal.  A  mental  model  may 
be  described  as  mental  representation  of  the  structure  and  behavior  of  a  system;  its  use  can  aid  the  user  in  explaining, 
predicting,  and  controlling  the  system’s  behavior  (Kramer  and  Schumacher,  1987,  cited  in  Webb  and  Kramer,  1990). 

Cognitive  skills  are  usually  combined  with  low-level  skills  and  may  be  used  in  complex  ways  when  solving  problems.  The 
transitions  between  plans,  strategic  knowledge,  and  mental  models  are  not  linear.  For  example,  an  individual  may  begin  to 
plan,  identify  one  or  more  sub-goals,  access  a  mental  model  to  determine  functional  relationships,  and  then  solve  the 
problem  associated  with  some  of  the  related  sub-goals.  Then,  the  individual  may  return  to  the  planning  activity  to  identify 
and  address  more  goals  and  sub-goals. 

4.3.4  Human  Error 

The  following  briefly  discusses  human  error;  a  more  detailed  description,  applicable  to  NPPs  and  other  complex  human- 
machine  systems,  is  provided  in  O’Hara  (1994).  When  cognitive  resources  are  not  used  effectively,  or  when  they  become 
depleted,  human  performance  suffers.  Human  error  occurs  when  a  person  does  not  perform  a  necessary  action,  or  performs 
an  inappropriate  one.  When  a  person  does  not  perform  a  needed  action  within  the  required  time,  it  is  sometimes  called  an 
error  of  omission.  When  a  person  performs  an  inappropriate  action,  it  is  sometimes  referred  to  as  an  error  of  commission. 
There  have  been  many  attempts  to  define  the  mechanisms  causing  errors.  One  widely  accepted  scheme  classifies  human 
errors  into  two  major  categories:  mistakes  and  slips'  (Norman,  1983  and  1988;  Lewis  and  Norman,  1986;  Reason,  1990). 
This  distinction  is  based  on  consideration  of  intention,  a  high-level  specification  of  actions  which  starts  a  chain  of 
processing  that  normally  results  in  the  actions  being  carried  out. 

An  error  in  intention  formation,  such  as  forming  an  inappropriate  plan,  is  called  a  mistake.  Mistakes  are  related  to 
incorrectly  assessing  the  situation  or  inadequately  planning  a  response.  When  a  plant  is  upgraded,  mistakes  can  occur  when 


'  The  category  of  errors  described  by  Reason  (1990)  as  lapses  is  not  discussed  here. 
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personnel  plan  inappropriate  actions  because  they  do  not  accurately  understand  the  new  systems  or  HSI  componenu.  Slips 
are  errors  in  carrying  out  intentions,  during  which  a  person  intends  to  do  one  thing  but  accomplishes  another.  Slips  result 
from  “automatic”  human  behavior,  when  subconscious  patterns  of  actions  (schemas)  intended  to  accomplish  the  intention, 
get  waylaid  en  route  to  execution  (Norman,  1983,  1988^  Thus,  slips  tend  to  occur  with  skilled  users,  rather  than  beginners 
learning  new  activities.  The  highly  practiced  behavior  of  an  expert  leads  to  the  lack  of  focused  attention  and  increases  the 
likelihood  that  slips  will  occur.  This  lack  of  attention  results  in  the  incorrect  activation  and  triggering  of  schemas. 

Many  types  of  slips  have  been  identified  (Norman,  1981,  1983,  1988;  Lewis  and  Norman,  1986).  A  review  of  computer- 
based  HSls  used  in  process  control  and  other  complex  systems  found  that  soft  controls  are  particularly  susceptible  to  the 
types  of  slips  listed  in  Table  4.1  (Stubler,  O’Hara,  and  Kramer,  2000);  their  likelihood  may  be  increased  for  upgrades  that 
include  soft  controls. 


Table  4,1  Types  of  Slips  That  May  Affect  Soft  Control  Usage 


Capture  Error:  An  error  of  execution  (slip)  that  occurs  when  an  infrequently  performed  action  requires  a 
sequence  of  operations,  some  of  which  are  the  same  as,  or  similar  to,  those  of  a  frequently  performed  action.  In 
attempting  the  infrequent  action,  the  more  frequent  action  is  performed  instead.  For  example,  an  operator 
intends  to  perform  task  1,  composed  of  operations  A,  B,  C,  and  D,  but  instead,  executes  the  more  frequently 
performed  task  2,  composed  of  operations  A,  B,  C,  and  E. 

Description  Error:  An  error  of  execution  (slip)  that  involves  performing  the  wrong  set  of  well-practiced 
actions.  Description  errors  occur  when  the  information  that  activates  or  triggers  the  action  is  either  ambiguous 
or  undetected. 


Loss-of-Activation  Error:  A  slip  that  occurs  when  an  intended  action  is  not  carried  out  due  to  a  failure  of 
memory  (i.e.,  the  intention  has  partially  or  completely  decayed  from  memory).  A  special  case  of  loss-of- 
activation  errors  involves  forgetting  part  of  an  intended  act  while  remembering  the  rest  (e.g.,  retrieving  a 
display  while  not  being  able  to  remember  why  it  is  needed). 

Misordered  Components  of  an  Action  Sequence:  A  slip  involving  skipped,  reversed,  or  repeated  steps.  Soft 
controls  may  be  prone  to  this  type  of  slip  because  they  require  sequential  operations  for  access  and  use. 

Mode  Error:  Performing  an  operation  that  is  appropriate  for  one  mode  when  the  device  is  in  another  mode. 
Mode  errors  occur  when  the  user  believes  the  device  is  in  one  mode  when  it  is  actually  in  another  and,  as  a 
result,  performs  an  inappropriate  action. 

Unintentional  Activation:  A  slip  that  occurs  when  a  set  of  actions  (schema)  that  is  not  part  of  a  current  action- 
sequence  becomes  activated  for  extraneous  reasons,  and  then  is  triggered,  leading  to  the  unintended  actuation  of 
an  input  device. _ 


4.4  Generic  Cognitive  Tasks 

Generic  cognitive  tasks  include  monitoring  and  detection,  situation  assessment,  response  planning,  and  response 
implementation  (O’Hara,  1994;  O’Hara,  Higgins,  Stubler,  and  Kramer,  2000).  These  cognitively  demanding  activities  are 
performed  by  personnel  in  their  roles  as  supervisory  controllers.  Any  primary  task  typically  can  be  described  in  terms  of 
one  or  more  of  these  generic  cognitive  tasks.  The  following  discusses  these  generic  cognitive  tasks,  including  how  the  basic 
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properties  of  human  information  processing  and  skilled  performance  (Section  4.3)  are  incorporated  into  them  and  also  how 
the  characteristics  of  upgrades  may  affect  them.  This  is  presented  first  in  terms  of  individual  performance,  and  then  in 
terms  of  a  crew’s  performance. 


4.4.1  Individual  Performance 

Monitoring  and  Detection 

Monitoring  and  detection  refer  to  the  activities  involved  in  extracting  information  from  the  environment,  such  as  checking 
the  plant’s  state  to  determine  whether  the  systems  and  equipment  are  operating  correctly.  This  can  include  monitoring 
parameters  indicated  on  the  CR  panels  and  displayed  by  the  process  computer,  obtaining  verbal  reports  from  operators  in 
different  areas,  and  sending  operators  to  areas  to  check  on  equipment.  Monitoring  also  includes  receiving  information  from 
other  personnel,  such  as  when  one  operator  listens  to  another  calling  out  values  of  requested  variables.  Detection  is  the 
operator’s  recognition  that  something  is  not  operating  correctly,  and  that  an  abnormality  exists.  Monitoring  and  detection 
are  influenced  by  two  factors:  the  characteristics  of  the  environment  (e.g.,  the  HSI),  and  a  person’s  knowledge  and 
expectations. 

In  a  CR  environment,  the  HSI  is  one  of  the  most  important  sources  of  information.  Monitoring  relies  on  the  use  of  HSI 
characteristics  that  support  the  identification  of  information  sources;  this  is  important  because  personnel  must  be  able  to 
find  the  right  indicator  or  display  before  it  can  be  read.  Some  HSI  characteristics  supporting  identification  include  display 
navigation  mechanisms  and  labels  for  indications.  The  inadequate  or  inappropriate  use  of  such  characteristics  can  result  in 
an  operator  selecting  the  wrong  indicator  or  display.  Capture  errors  and  description  errors  are  two  examples  of  slips  that 
may  result  in  wrong  choices  (Stubler,  O’Hara,  and  Kramer,  2000). 

Detection  is  influenced  by  HSI  characteristics  which  affect  human  perceptual  processes  and  support  personnel  in  noticing 
the  information.  Examples  include  display  formats,  such  as  digital  readouts  and  graphical  (analog)  representations,  and 
coding  schemes,  such  as  color  and  shapes  that  affect  the  salience  of  information. 

The  HSI  characteristics  that  support  monitoring  and  detection  interact  with  people’s  knowledge  and  expectations.  This 
interaction  affects  their  selective  attention  capabilities,  producing  data-driven  and  model-driven  monitoring  (described  in 
Section  4.3.2).  For  example,  in  NPP  CRs,  alarm  systems  are  basically  automated  monitors  that  influence  data-driven 
monitoring  using  visual  and  auditory  salience  to  draw  attention.  Auditory  alerts,  flashing,  and  color  coding  are  some 
physical  characteristics  that  enable  operators  to  quickly  identify  important  alarms.  Data-driven  monitoring  depends  on  the 
relative  and  absolute  levels  of  salience.  The  strength  of  a  stimulus  must  exceed  some  threshold  to  draw  attention.  Above 
that  value,  personnel  tend  to  notice  stimuli  with  the  greatest  salience. 

Model-driven  monitoring  is  directed  by  people’s  knowledge  and  expectations.  It  is  active  monitoring  in  the  sense  that 
personnel  deliberately  direct  attention  to  areas  they  expect  to  have  useful  information,  rather  than  merely  attending  to 
stimuli  based  on  their  relative  salience.  Model-driven  monitoring  may  be  initiated  by  several  factors.  First,  it  may  be 
guided  by  operating  procedures  or  standard  practice  (e.g.,  control  panel  walk-downs  that  accompany  shift  turnovers). 
Second,  it  can  be  triggered  by  situation  awareness  or  response  planning  activities  and,  therefore,  is  strongly  influenced  by 
the  individual’s  understanding  of  the  current  situation.  This  understanding  allows  the  operator  to  direct  attention  more 
effectively  by  focusing  monitoring  activities  on  information  sources  that  are  likely  to  change  or  provide  useful  information. 
However,  model-driven  monitoring  can  also  lead  operators  to  miss  important  information.  For  example,  incorrect 
understanding  may  cause  an  operator  to  focus  attention  in  the  wrong  place,  to  fail  to  observe  a  critical  finding,  or  to 
misinterpret  or  discount  an  indication. 
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Operators  in  a  NPP  CR  may  be  faced  with  an  information  environment  containing  more  variables  than  can  be  realistically 
monitored  at  one  time.  For  example,  during  an  early  transient,  they  may  be  confronted  by  an  overwhelming  number  of 
alarms  and  changing  parameters.  Due  to  constraints  in  human  information  processing,  the  number  of  indications  that 
operators  can  attend  to  at  any  given  time  or  maintain  in  working  memory  is  limited.  Attention  resources  can  be  used  more 
efficiently  by  mentally  combining  individual  indications  into  related  groups  using  strategies  for  information  chunking 
(Section  4.3.2).  The  ability  to  chunk  information  can  be  strongly  affected  by  the  way  it  is  presented.  Display  formats, 
physical  proximity,  and  coding  methods  can  enhance  or  impair  the  ability  of  people  to  perceive  patterns  and  relationships. 
These  factors  can  produce  emergent  features  (Wickens  and  Carswell,  1997)  that  convey  higher-level  information.  For 
example,  if  all  the  needles  on  a  set  of  indicators  normally  point  in  one  direction,  then  the  presence  of  a  needle  pointing  in 
another  direction  conveys  at  a  glance  the  higher-level  message  that  status  is  normal  for  all  indicators  except  the  one  with 
the  abnormal  orientation.  (The  application  of  emergent  features  and  other  concepts  to  designing  computer-based  displays 
is  discussed  in  O’Hara,  Higgins,  and  Kramer,  2000.) 

Information  chunking  is  also  affected  by  the  operator’s  mental  models.  Some  chunks  are  static;  the  relationships  do  not 
change  with  changes  in  plant  status.  For  example,  functional  relationships  based  on  the  structure  of  the  plant  do  not 
change.  Other  relationships  between  individual  indications  change  based  on  the  operator’s  understanding  of  the  situation. 
For  example,  a  set  of  indications  may  be  grouped  together  if  they  are  thought  to  be  related  to  a  common  cause,  such  as  a 
single  malfunction  or  the  plant’s  operating  mode. 

CR  personnel  may  also  have  to  filter  information  by  applying  selective  attention  to  the  most  relevant  indications.  Other 
indications  may  be  ignored  or  relegated  to  a  lower  status  that  is  monitored  when  resources  permit  (Mumaw  et  al.,  1994). 
This  strategy  requires  the  operator  to  decide  what  to  monitor  and  when.  Filtering  strategies  are  related  to  the  operator’s 
understanding  of  the  current  situation,  which  guides  the  allocation  of  attention  resources  for  sampling  data  from  the 
environment  based  on  statistical  properties.  These  include  the  expected  probability  that  useful  information  will  be  found 
and  the  correlation  of  related  indications  (e.g.,  the  way  that  one  variable  behaves  in  relationship  to  others).  Filtering  can  be 
part  of  an  iterative  monitoring  process.  An  initial  indication  of  an  anomaly  can  cause  operators  to  adjust  their 
understanding  of  the  current  situation,  which  affects  the  way  future  indications  are  filtered.  Thus,  the  design  characteristics 
of  the  HSI  that  convey  information  about  plant  status  to  the  operator,  such  as  alarms  and  coding  of  abnormal  indications, 
can  significantly  affect  the  use  of  appropriate  filtering  strategies. 

During  an  HFE  review,  attention  should  be  given  to  HSI  upgrades  that  change  the  way  that  information  is  presented  and, 
thus,  affect  monitoring  and  detection,  including  low-salience  characteristics  that  reduce  the  detectability  of  indications  and 
high-salience  ones  that  encourage  data-driven  monitoring.  The  characteristics  of  information  display  characteristics  that 
affect  model-driven  monitoring  also  should  be  considered.  This  includes  presentation  modes  that  influence  people’s  ability 
to  understand  the  current  state  of  the  plant,  such  as  the  overall  status  of  systems.  It  also  includes  characteristics  of 
personnel’s  ability  to  recognize  relationships  between  conceptually  related  indications.  Failures  in  monitoring  can  include 
failing  to  observe  parameters,  observing  the  wrong  ones,  misunderstanding  their  significance,  or  failing  to  obtain  needed 
information  from  operators  elsewhere  in  the  plant.  Failures  in  detection  can  include  failure  to  recognize  an  abnormality 
despite  appropriate  monitoring.  An  error  in  monitoring  or  in  detection  can  cause  the  operator’s  failure  to  respond  to  the 
event  within  the  requisite  time. 

Situation  Assessment 


Situation  assessment  refers  to  the  cognitive  activities  involved  in  processing  stimuli  to  construct  an  understanding  of  the 
situation.  Two  important  ones  are  interpreting  the  current  state  and  determining  its  implications  (Mumaw  et  al.,  1994). 
Interpreting  the  current  state  entails  developing  a  mental  representation  of  the  plant’s  status  based  on  information  from 
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monitoring  and  detection.  This  activity  is  affected  by  limitations  and  biases  in  the  human  information  processing  system. 
For  example,  “garden  path”  interpretations  may  occur  when  the  symptoms  of  the  plant’s  current  condition  resemble  those  of 
a  familiar  or  well-learned  event.  Then,  this  initial  correspondence  influences  the  operator’s  expectations,  leading  to  an 
incorrect  diagnosis.  Interpreting  the  current  state  involves  skills  for  properly  assessing  different  types  of  information, 
including  that  which  confirms,  disconfirms,  or  differentiates  it  from  information  gathered  earlier.  Humans  in  dynamic 
information  environments  are  biased  toward  seeking  only  confirmations  (e.g.,  Evans,  1989,  cited  in  Mumaw  et  al.,  1994); 
that  is,  they  are  likely  to  discount  information  that  contradicts  their  current  interpretation  of  the  plant’s  state  or  that 
supports  other  interpretations.  This  is  called  the  confirmation  bias.  It  poses  the  risk  that  personnel  will  develop  incorrect  or 
incomplete  interpretations  (Mumaw  et  al.,  1994). 

Determining  the  implications  of  the  current  state  entails  “mental  simulation”  in  which  future  states  are  anticipated,  goals 
for  action  identified,  and  priorities  for  goals  established.  Mental  simulation  relies  on  mental  representations  of  the  plant 
and  the  current  situation.  Deficiencies  in  these  representations  can  result  in  inaccurate  assessments  of  implications.  These 
mental  representations  involve  two  related  concepts;  the  mental  model  and  the  situation  model.  Each  is  described  below. 

Mental  Model  -lY\e  mental  model  of  the  plant  is  the  operator’s  internal  representation  of  the  physical  and  functional 
characteristics  of  the  plant  and  its  operation.  An  operator  may  possess  multiple  mental  models  of  different  parts  of  the  plant 
at  different  levels  of  abstraction.  They  are  developed  through  training  and  experience  and  are  reinforced  by  the  way 
information  is  displayed  in  the  HSI,  such  as  instrument  layouts  depicting  relationships  between  components  in  a  system. 
Mental  models  are  not  always  fully  accurate  or  complete.  When  plants  are  upgraded,  personnel  may  need  to  modify  their 
mental  models  to  accommodate  the  changes. 

Changes  in  systems  or  the  HSI  can  add  new  levels  of  complexity  that  must  be  incorporated  into  the  operator’s  mental 
model.  Systems  often  have  complicated  interconnections  with  other  systems  and  variables.  With  the  introduction  of  digital 
I&C  technology,  these  interconnections  may  become  even  more  complex.  That  is,  with  analog  control  systems,  the 
connections  between  control  devices  and  the  equipment  tend  to  be  rather  direct.  However,  with  digital  I&C  systems,  these 
connections  may  become  more  complicated  (e.g.,  include  data  buses,  signal  multiplexers,  and  intermediate  processors).  In 
addition,  introducing  more  automation  in  systems  greatly  increases  their  complexity,  making  them  more  difficult  to 
understand.  This  fact  was  confirmed  during  site  visits  to  foreign  NPPs  (O’Hara,  Stabler,  and  Higgins,  1996).  Training 
personnel  stated  that  for  the  newer  plants,  which  feature  more  automation,  the  operators  had  to  learn  much  more 
information  than  operators  of  the  older  plants  to  have  equivalent  understandings  of  the  plant’s  structure  and  functions. 

HSI  upgrades,  such  as  advanced  display  systems,  alarm  systems,  computerized  operator  support  systems,  and  computer- 
based  procedure  systems,  can  support  the  formation  of  the  operator’s  mental  model  of  plant  systems.  For  example, 
graphical  displays  can  depict  complicated  functional  or  physical  relationships  between  plant  components,  reinforcing  the 
operator’s  understanding.  However,  HSI  upgrades  can  also  impose  new  demands  on  the  personnel  by  increasing  HSI 
complexity  (e.g.,  introducing  computer-based  display  systems  and  new  methods  of  interaction).  New  information  about  the 
HSI  (e.g.,  the  organization  of  display  structures  and  the  new  methods  of  interaction)  must  be  incorporated  into  the 
operator’s  mental  model  of  the  HSI. 

Situation  model  -  The  situation  model  is  an  understanding  of  the  specific  current  situation.  It  is  based  on  factors  known,  or 
hypothesized,  to  be  affecting  the  situation  at  a  given  time.  To  construct  a  situation  model,  operators  use  their  general 
knowledge  and  understanding  about  the  plant  and  its  operation  to  interpret  the  information.  The  situation  model  is  affected 
by  the  way  the  plant’s  behavior  is  represented  by  the  HSI  and  is  constantly  updated  as  new  information  is  received;  any 
limitations  in  general  or  specific  information  may  result  in  incomplete  or  inaccurate  situation  models. 
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The  ability  to  form  an  adequate  situation  model  may  be  negatively  affected  by  changes  in  either  plant  systems  or  the  HSI. 
Changes  in  systems,  such  as  installing  digital  instrumentation  and  control  (I&C)  systems,  can  change  the  plant’s  behavior, 
such  as  by  increasing  or  decreasing  a  system’s  response  time.  Introducing  additional  automation  can  introduce  new  control 
modes,  which  operators  must  monitor  to  determine  whether  they  are  functioning  properly  and  satisfying  plant  goals.  Such 
changes  add  new  dimensions  in  plant  performance  that  must  be  addressed  by  the  situation  models.  Situation  models  must 
also  account  for  the  status  of  the  HSI;  operators  must  consider  possible  faults  in  the  HSI  when  interpreting  abnormal  plant 
indications.  In  addition,  when  HSI  upgrades  involve  higher  levels  of  automation,  operators  must  consider  its  behavior  in 
their  situation  model.  For  example,  computer-based  display  systems  and  decision  aids  may  automatically  retrieve  and 
present  information  to  the  operator,  based  on  their  interpretations  of  plant  conditions.  Cognitive  demands  may  be  imposed 
on  the  operator  in  understanding  how  these  systems  operate  (e.g.,  why  a  particular  piece  of  information  was  displayed),  and 
anticipating  their  future  behavior  as  conditions  change. 

Inadequacies  in  situation  awareness  may  cause  potential  threats  to  plant  safety  to  go  unrecognized.  As  a  result,  operators 
may  fail  to  take  action  early  in  a  developing  transient,  and  so  conditions  worsen.  The  plant’s  state  may  be  misdiagnosed 
due  to  misinterpretation  of  symptoms.  Such  failures  in  situation  awareness  can  affect  the  appropriateness  or  adequacy  of 
response  planning  (discussed  next),  thus  generating  errors  (i.e.,  mistakes).  For  example,  as  a  result  of  misdiagnosis, 
operators  may  plan  a  response  to  a  particular  fault  condition  when  a  different  one  exists.  Alternatively,  operators  may 
correctly  diagnose  the  fault  but  then  select  actions  that  caimot  be  successfully  completed  under  the  plant’s  current 
configuration  of  the  plant  (e.g.,  a  necessary  system  is  not  available). 

Response  Plannin£ 

Response  planning  refers  to  the  cognitive  task  of  developing  an  approach  for  achieving  a  goal.  The  planned  response  may 
be  as  simple  as  deciding  to  access  a  particular  parameter  from  a  particular  HSI  component,  or  it  may  involve  selecting  a 
complicated  course  of  action,  guided  by  a  procedure.  Response  planning  based  on  procedures,  established  practices,  or 
heuristics  is  referred  to  as  rule-based  decision  making  (Rasmussen,  1986).  In  some  cases,  personnel  may  be  forced  to  make 
decisions  for  situations  that  are  not  explicitly  or  completely  addressed  by  procedures.  Then,  they  must  develop  plans  based 
on  their  knowledge  of  the  plant,  the  situation,  and  possible  success  paths;  this  is  referred  to  as  knowledge-based  decision 
making  (Rasmussen,  1986).  Roth  et  al.  (1994)  describe  the  use  of  knowledge-based  behavior  during  simulations  in  which 
the  complexities  of  the  interfacing  system  loss-of-coolant  accident  (ISLOCA)  scenarios  made  it  difficult  for  operators  to 
merely  follow  the  procedures.  Knowledge-based  decision  making  is  especially  important  during  severe  accident 
management  (e.g.,  Mumaw  et  al.,  1994)  in  which  conditions  may  extend  beyond  those  explicitly  addressed  by  the 
emergency  operating  procedures. 

Response  planning  involves  using  one’s  situation  model  to  select  goals  and  one’s  mental  model  of  the  plant  to  identify  paths 
for  achieving  them.  This  can  place  high  demands  on  cognitive  resources,  such  as  working  memory,  long-term  memory,  and 
attention,  especially  in  unfamiliar  situations,  when  procedures  do  not  appear  to  be  adequate.  Individual  skills  involved  in 
response  planning  include  the  ability  to  identify  appropriate  existing  plans  (such  as  a  particular  emergency  operating 
procedure),  formulate  a  response  plan,  evaluate  a  response  plan,  and  determine  the  sequence  of  actions  required  to  execute  a 
plan  (Mumaw  et  al.,  1994). 

These  response-planning  skills  may  be  strongly  influenced  by  features  of  the  HSI.  For  example,  identifying  a  response  plan 
depends  on  whether  personnel  can  maintain  awareness  of  the  plant’s  changing  state,  identify  the  highest  priority  goals,  and 
evaluate  the  suitability  of  existing  procedures  in  terms  of  available  time,  required  availability  of  plant  systems,  and  required 
entry  conditions.  This  ability  may  be  supported  by  HSI  components,  such  as  alarms  and  displays,  that  enhance  an 
operator’s  situation  awareness  and  support  identifying  goal  conflicts,  and  decision  aids  that  support  the  evaluation  of 
existing  procedures  for  current  conditions.  Formulating  and  evaluating  response  plans  are  also  supported  by  these  features. 
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and  may  be  extended  by  HSI  features  that  support  the  assessment  of  potential  consequences  and  side  effects  of  various 
response  paths.  Response  planning  may  be  negatively  affected  by  decision  aids  with  limited  or  incomplete  guidance  or  that 
lack  mechanisms  for  examining  the  rationale  and  limitations  of  the  guidance. 

Failures  in  response  planning  result  in  the  development  of  inappropriate  plans  that,  if  executed,  may  be  described  as 
mistakes  (Section  4.3.4).  Inappropriate  plans  may  have  a  range  of  consequences.  For  example,  those  that  produce 
unnecessary  but  inconsequential  actions  are  generally  ignored  in  an  error  analysis.  They  also  may  produce  inefficiencies 
that  are  of  little  consequence.  For  example,  the  desired  goal  may  be  achieved  but  with  a  greater  expenditure  of  resources 
(e.g.,  time  and  personnel)  than  by  other  available  paths.  Finally,  some  inappropriate  plans  may  have  significant  negative 
consequences,  such  as  the  violation  of  a  limit  set  by  technical  specifications,  loss  of  a  safety  function,  or  the  failure  to 
effectively  isolate  a  contamination  source.  It  is  these  mistakes  that  are  of  greatest  concern  for  response  planning. 

When  conducting  HFE  reviews  of  NPP  upgrades,  consideration  should  be  given  to  changes  in  a  plant’s  functions  that  create 
new  demands  for  response  planning,  and,  therefore,  new  opportunities  for  mistakes.  Attention  also  should  be  directed 
toward  HSI  characteristics  such  as  display  design  features,  procedures,  and  decision  aids,  that  may  affect  the  ability  of 
personnel  to  make  appropriate  plans. 

Response  Implementation 

Response  implementation  is  the  performance  of  the  actions  identified  during  response  planning.  This  can  be  as  simple  as 
selecting  and  operating  a  single  control,  or  as  complicated  as  executing  a  control  sequence  involving  multiple  controls  and 
displays  and  personnel.  The  HSI’s  design  determines  where  and  how  these  actions  can  occur,  and  where  and  how  feedback 
can  be  monitored. 

Two  aspects  of  NPPs  that  can  complicate  response  implementation  are  response  time  and  the  directness  of  observation. 
Delays  in  response  time  and  the  accompanying  feedback  disrupt  implementation  because  they  make  it  difficult  to  determine 
whether  control  actions  are  having  their  intended  effect.  The  operators  then  may  have  to  rely  on  predicting  future  states 
based  on  their  mental  models  of  the  plant’s  behavior.  This  prediction  is  prone  to  errors  (i.e.,  mistakes)  due  to  the 
complexity  of  system  behavior.  For  example,  errors  in  predicting  a  system’s  response  may  result  in  operators  providing 
inputs  that  cause  the  system  to  overshoot  or  undershoot  the  target  value,  generating  a  transient  in  which  the  parameter 
swings  widely  outside  acceptable  control  ranges.  Thus,  the  task  of  implementing  a  response  can  be  negatively  affected  by 
changes  in  the  plant’s  systems  that  delay  response  times. 

While  eliminating  such  delays  can  avoid  problems  associated  with  predicting  the  system’s  behavior,  a  fast  response  can 
pose  other  problems.  When  control  systems  are  upgraded  (e.g.,  with  digital  systems),  their  speed  may  increase  to  the  point 
that  they  can  react  almost  immediately  to  inputs  from  operators.  This  can  create  opportunities  for  errors  of  execution  (i.e., 
slips).  A  review  of  industrial  experience  with  digital  systems  found  that,  in  some  cases  when  an  operator  made  an  incorrect 
entry,  these  systems  responded  so  quickly  that  the  operator  did  not  have  sufficient  time  to  correct  it,  nor  halt  the  system’s 
response  (Stubler,  O’Hara,  and  Kramer,  2000).  For  example,  if  an  incorrectly  entered  setpoint  differed  greatly  from  the 
previous  value,  then  system  might  change  rapidly,  causing  a  transient  and  damaging  equipment.  Thus,  increases  in  the 
speed  of  plant  systems  can  have  negative  affects  on  the  response-implementation  task  if  features  for  preventing,  detecting, 
and  correcting  errors  are  inadequate. 

The  directness  of  observation  refers  to  the  fact  that  operators  usually  cannot  directly  observe  the  effects  of  their  control 
actions  on  systems.  Instead,  they  must  be  inferred  from  the  HSI’s  displays.  The  ability  of  people  to  make  inferences  is 
affected  by  the  way  information  is  presented.  For  example,  the  operator  may  not  be  able  to  interpret  the  status  of  plant 
systems  if  the  appropriate  display  cannot  be  quickly  accessed,  nor  if  the  display  is  not  well  suited  to  this  task.  Thus, 
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display-design  characteristics  of  HSI  upgrades  can  have  important  effects  on  the  operator’s  implementation  response 
(O’Hara,  Higgins,  and  Kramer,  2000). 

Failures  in  response  implementation  can  lead  to  the  operation  of  the  wrong  equipment,  or  the  incorrect  operation  of  the 
correct  piece  of  equipment.  If  an  upgrade  changes  a  system’s  response  time,  then  the  potential  effects  on  personnel’s 
performance,  including  errors  resulting  from  either  an  increase  or  a  decrease,  should  be  assessed.  If  an  upgrade  will  change 
the  way  information  is  presented  via  the  HSI,  these  potential  effects  also  should  be  assessed.  These  assessments  should 
include  the  ability  of  personnel  to  select  the  correct  control,  execute  the  control  action  properly,  and  correctly  evaluate  the 
effects  of  the  action.  Particular  attention  should  be  given  to  tasks  that  are  potentially  safety  significant. 

4.4.2  Crew  Performance  and  Team  Skills 

A  team  may  be  defined  as  a  set  of  two  or  more  people  who  interact,  dynamically,  interdependently,  and  adaptively  toward  a 
common  and  valued  goal,  objective,  or  mission,  each  having  specific  roles  or  frmctions  to  jjerform  (Salas  et  al.,  1992).  In  a 
study  of  team  performance  in  naval  tactical  training,  Oser  et  al.  (1989)  identified  the  following  behaviors  that  were 
important  to  performance:  identification  and  resolution  of  errors,  coordinated  information  exchange,  and  team 
reinforcement.  Successful  teams  actively  located  errors,  questioned  improper  procedures,  and  monitored  the  status  of 
others.  A  NPP  operating  crew  is  a  team  in  which  the  members  share  information  and  perform  their  tasks  in  coordination  to 
satisfy  specific  operational  and  safety  goals.  The  crew  members  interact  with  each  other  and  with  other  personnel  in  the 
plant.  The  HSI  provides  the  context  within  which  personnel  convey  information  and  coordinate  actions.  The  four  generic 
cognitive  tasks  (i.e.,  monitoring  and  detection,  situation  awareness,  response  plaiming,  and  response  implementation)  each 
require  interactions  between  crew  members.  The  following  discusses  these  tasks  from  the  perspective  of  crew  performance, 
and  how  performance  may  be  affected  by  upgrades. 

Monitoring  and  Detection 

The  ability  of  crew  members  to  communicate  information  about  the  plant’s  status  to  the  primary  decision  maker  (e.g., 
senior  reactor  operator  or  shift  supervisor)  is  a  major  consideration  of  monitoring  and  detection.  Operators  must 
communicate  in  a  way  that  allows  decision  makers  to  rapidly  incorporate  this  information  into  their  developing  situation 
models.  Communication  failures  can  have  many  causes.  Cognitive  causes  stem  from  a  lack  of  a  common  understanding. 
Much  spoken  communication  relies  on  a  shared  context  between  the  speaker  and  the  listener  to  support  the  correct 
interpretation  of  the  intended  meaning.  Typical  failures  involve  hidden  differences  in  fi^es  of  reference  and  assumptions 
(Mumaw  et  al.,  1994).  Communication  may  be  impaired  to  the  degree  that  personnel  have  different  situation  models.  For 
example,  due  to  differences  in  their  understanding  of  the  plant’s  state,  an  operator  may  misinterpret  a  request  for 
information  from  the  decision  maker,  or  the  decision  maker  may  misinterpret  information  provided  by  an  operator. 
Communication  may  also  be  affected  by  physical  considerations.  For  example,  upgrades  can  change  operators’  tasks,  thus 
increasing  the  physical  distance  between  operators  (e.g.,  an  operator  may  be  required  to  spend  more  time  at  back  panels,  or 
away  from  the  main  control  panel).  Other  physical  considerations  include  modifications  that  affect  the  line-of-sight 
between  operators,  and  ones  that  increase  background  noise  making  speech  less  intelligible. 

Situation  Awareness 


During  a  transient,  the  primary  decision  maker  attempts  to  develop  a  coherent  understanding  of  changes  in  the  plant’s 
status  from  available  information.  Due  to  limited  cognitive  resources  for  seeking  information  and  drawing  inferences,  the 
decision  maker  relies  on  other  crew  members  for  input.  Operators  in  the  CR  are  likely  to  develop  different  interpretations 
of  the  plant’s  state  from  the  information  they  access,  and  differences  in  their  individual  representations  of  system  status. 
However,  if  crew  members  are  aware  of  the  decision  maker’s  interpretation  of  events,  they  can  seek  confirming  and 
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disconfiming  evidence.  Thus,  the  interpretation  of  plant  status  depends  upon  the  ability  of  decision  makers  to  convey  their 
interpretations,  and  the  ability  of  crew  members  to  communicate  their  findings  and  interpretations.  Another  element  of 
situation  awareness,  determining  the  implications  of  plant  state,  may  require  the  decision  maker  to  mentally  simulate  events 
to  predict  future  states.  Team  performance  depends  upon  the  decision  maker’s  ability  to  communicate  the  intermediate 
results  of  these  mental  simulations  to  crew  members  so  they  may  provide  input  (Mumaw  et  al.,  1994). 

Response  Plannin2 

During  response  planning,  the  decision  maker  formulates  plans  based  on  specific  goals.  Here,  interactions  between 
personnel  are  important,  especially  when  the  situation  cannot  be  addressed  by  straightforwardly  applying  an  existing 
procedure  (e.g.,  incidents  not  explicitly  addressed  by  emergency  operating  procedures;  severe  accidents).  One  important 
skill  is  the  ability  of  the  decision  maker  to  communicate  the  intermediate  results  of  response  planning  to  others,  so  they  may 
help  evaluate  it  and  contribute  additional  information.  Another  is  the  ability  of  crew  members  to  convey  evidence  to  the 
decision  maker  showing  that  the  plan  is  no  longer  feasible,  such  as  when  plant  conditions  change  quickly  (Mumaw  et  al 
1994). 

Response  Implementation 

Response  implementation  requires  the  coordinated  actions  of  crew  members  to  execute  the  selected  plan.  People  must 
maintain  awareness  of  each  other’s  actions,  so  that  parallel  activities  are  coordinated.  In  addition,  operators  may  monitor 
each  other’s  actions  to  detect  errors  in  responding. 

Crew  performance  is  supported  when  personnel  can  develop  a  shared  view  of  the  plant’s  status  and  CR  activities.  This  may 
be  supported  by  HSI  characteristics  that  provide  them  with  access  to  similar  sets  of  information  about  plant  conditions, 
support  their  ability  to  communicate  plant  information  to  each  other  effectively,  support  coordination  of  activities  by 
allowing  them  to  maintain  an  awareness  of  each  other’s  actions,  and  support  monitoring  of  personnel’s  performance  to 
detect  errors.  Hutchins  (1990)  identified  three  characteristics  of  the  work  environment  that  support  team  performance: 

•  Horton  of  Observation  -  This  refers  to  the  portion  of  the  team’s  task  that  can  be  seen  or  heard  by  each  individual. 
It  results  from  the  arrangement  of  the  work  environment  (e.g.,  proximity  of  team  members)  and  is  influenced  by 
the  openness  of  tools  and  the  openness  of  interactions.  By  making  portions  of  a  task  observable,  other  team 
members  can  monitor  for  errors  of  intent  and  execution,  and  for  situations  in  which  additional  assistance  may  be 
helpful. 

•  Openness  of  Tools  —  This  refers  to  the  degree  to  which  an  observer  is  able  to  infer  information  about  another  crew 
member’s  ongoing  tasks  by  observing  a  tool’s  use.  Open  tools  show  characteristics  of  the  problem  domain  that 
provide  an  observer  with  a  context  for  understanding  what  has  been  done,  and  the  possible  implications.  For 
example,  a  control  task  carried  out  by  directly  manipulating  a  graphical  interface  (e.g.,  mimic  representation  of  a 
system)  may  be  more  useful  for  an  observer  (by  showing  which  component  is  being  operated  and  the  effect  on  the 
rest  of  the  system)  than  if  the  same  task  was  done  by  other  means,  such  as  with  text  commands  via  a  keyboard. 

•  Openness  of  Interaction  -  This  refers  to  the  degree  to  which  the  interactions  between  team  members  provide  an 
opportunity  for  others  with  relevant  information  to  contribute.  Openness  of  interaction  depends  on  the  type  of 
communication  (e.g.,  discussing  actions  or  decisions  in  the  presence  of  others)  and  the  style  of  interaction  (e.g.,  the 
degree  to  which  unsolicited  input  is  accepted).  Openness  of  interaction  is  also  influenced  by  characteristics  of  the 
work  environment  (e.g.,  openness  of  tools,  horizon  of  observation)  that  provide  other  team  members  with  an 
opportunity  to  see  and  hear  the  interaction. 
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Conventional  control  room  designs  typically  provide  a  broad  horizon  of  observation  that  facilitates  the  observation  of  team 
activities  (Stubler  and  O’Hara,  1996b).  Conventional  hardwired  controls  and  displays  may  be  “open  tools”  in  the  sense  that 
an  observer  can  infer  information  about  actions  (e.g.,  which  plant  system  was  involved,  which  control  was  operated,  what 
action  was  taken)  by  observing  the  operator’s  location  at  a  control  panel  and  the  action  performed.  Interactions  may  be 
considered  “open”  when  most  are  verbal  ones  that  can  be  heard  by  others  in  the  control  room.  Practices  such  as  calling  out 
requests  for  information  and  calling  back  the  responses  allow  other  control  room  personnel  to  maintain  awareness  of 
operational  activities. 

The  crew’s  performance  may  be  impaired  when  upgrades  change  these  HSI  features  and  operational  practices,  as  may  occur 
when  the  display  or  interaction  features  of  HSI  upgrades  interfere  with  the  ability  of  personnel  to  share  a  view  of  the  plant. 
An  example  is  an  individual  operator  console,  which  gives  individual  views  of  the  plant  that  are  not  easily  seen  by  others. 

If  other  crew  members  cannot  view  similar  information,  coordination  difficulties  may  eirise.  For  example,  when  interacting 
with  the  operator,  greater  opportunities  for  failures  in  communication  may  occur  because  other  crew  members  do  not 
understand  the  operator’s  frame  of  reference.  If  such  individual  consoles  are  used  to  execute  control  actions,  error  detection 
may  be  compromised  because  other  crew  members  may  not  be  able  to  observe  those  actions  while  performing  their  own. 
Thus,  subtle  mechanisms  for  maintaining  awareness  of  control  actions  and  detecting  errors  may  be  lost  (Stubler  and 
O’Hara,  1996b).  A  related  example  is  decision  aids  that  provide  special  information  to  an  individual,  but  do  not  make  it 
accessible  to  the  crew.  Consequently,  the  user  may  experience  difficulty  in  communicating  complicated  concepts  to  the  rest 
of  the  crew  because  members  do  not  share  a  common  view  of  plant  conditions. 

The  physical  characteristics  of  HSI  upgrades  also  can  affect  the  crew’s  performance.  For  example,  face-to-face 
communications  may  be  impaired  by  factors  that  interfere  with  their  line-of-sight,  or  new  equipment  may  requires  operators 
to  spend  more  time  away  from  the  main  control  panel  (e.g.,  monitoring  back  panels).  Another  characteristic  of  the  HSI  that 
may  impair  crew  coordination  is  background  noise  that  can  affect  the  intelligibility  of  speech. 
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NUREG-071 1  provides  a  general  process  for  reviewing  HSIs.  However,  when  upgrades  are  made  to  an  existing  NPP, 
special  considerations  may  arise.  The  purpose  of  this  section  is  to  identify  issues  of  human  performance  associated  with 
HFE  reviews  of  upgrades,  and  to  establish  a  technical  basis  upon  which  to  develop  guidelines.  To  prepare  this  section,  it 
was  necessary  to  gather  and  integrate  information  from  many  sources,  including  the  nuclear  power  industry,  other  process 
industries,  and  commercial  aviation,  and  from  general  literature  addressing  human  performance  in  complex  human- 
machine  systems.  This  section  also  includes  findings  from  visits  to  a  broad  range  of  facilities,  which  were  in  made 
conjunction  with  this  and  other  research  projects. 

Human  performance  issues  are  presented  in  sections  organized  according  to  the  four  phases  of  NUREG-071 1 :  Section  5.1 
addresses  the  planning  phase.  Section  5.2,  the  analysis  phase,  Section  5.3,  the  interface  design  phase,  and  Section  5.4,  the 
evaluation  phase.  Specific  NUREG-071 1  elements  are  described  within  each  of  these  phases.  Finally,  Section  5.5  discusses 
the  implementation  of  upgrades,  including  considerations  that  may  affect  personnel’s  acceptance  of  them,  and  the 
successful  integration  of  upgrades  into  existing  work  environments. 

5.1  Planning  Phase 

5.1.1  General  Considerations 

The  overall  goal  of  an  HFE  review  should  be  to  ensure  that  the  development  of  the  upgrade  is  consistent  with  good  HFE 
principles  of  design.  Such  reviews  are  necessary  if  the  upgrade  (1)  affects  the  role  of  personnel  or  the  tasks  by  which  they 
perform  their  role,  and  (2)  is  potentially  significant  to  plant  safety.  Upgrades  affect  the  role  or  tasks  of  plant  personnel  if 
they  impose  new  or  different  demands  on  them  for  operation,  maintenance,  or  activities  related  to  plant  safety.  Determining 
the  safety  significance  of  upgrades  requires  considering  how  safety  may  be  affected  by  them.  The  following  are  three 
approaches  derived  from  existing  NRC  review  criteria.  Others  are  also  possible. 

The  first  approach  for  assessing  the  potential  safety  significance  of  an  upgrade  is  based  on  the  criteria  for  unreviewed  safety 
questions  (USQ).  A  proposed  change  to  a  NPP  is  considered  to  involve  a  USQ  if  it  satisfies  any  of  the  three  criteria  of 
10  CFR  50.59.  The  Electric  Power  Research  Institute  (EPRI)  produced  the  Guideline  on  Licensing  Digital  Upgrades,  EPRI 
TR- 102348,  (EPRI,  1993)  to  assist  licensees  in  licensing  digital  upgrades  using  these  criteria.  This  document  was  favorably 
reviewed  with  certain  specific  interpretations  by  the  NRC  in  Generic  Letter  95-02  (NRC,  1995d).  EPRI  TR- 102348  restates 
the  three  criteria  of  10  CFR  50.59  as  seven  questions,  presented  in  Table  5.1;  an  affirmative  answer  to  any  one  of  them 
indicates  that  a  USQ  exists.  An  affirmative  answer  is  not  a  determination  of  safety  signific2uice,  but  merely  an  indication 
that  there  is  a  question  related  to  safety  which  has  not  been  reviewed  by  the  NRC.  These  seven  questions  may  be  adopted  to 
assess  the  safety  of  upgrades.  An  example  of  an  evaluation  method  based  on  these  questions  is  given  by  Stubler,  et  al. 
(1996). 

A  second  approach  for  assessing  the  potential  safety  significance  of  an  upgrade  is  to  base  the  evaluation  on  criteria  from  the 
Maintenance  Rule,  NRC  Inspection  Procedure  62706  (NRC,  1995e),  which  imposes  special  requirements  for  monitoring  the 
performance  or  condition  of  structures,  systems,  and  components  (SSC)  that  may  be  considered  significant  to  safety.  In  the 
simplified  flowchart  of  the  Maintenance  Rule,  the  following  criteria  are  set  out  for  determining  whether  a  structure,  system, 
or  component  falls  within  its  scope.  An  SSC  is  considered  safety  significant  if  at  least  one  of  the  following  questions  is 
answered  affirmatively: 

•  Is  the  SSC  safety  related? 

•  Does  the  non-safety  related  SSC  mitigate  accidents  or  transients? 
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Table  5.1  Primary  USQ  Analysis  Questions  from  EPRI  TR-102348 


1. 

May  the  proposed  activity  increase  the  probability  of  occurrence  of  an  accident  evaluated  previously  in 
the  Safety  Analysis  Report  (SAR)? 

2. 

May  the  proposed  activity  increase  the  consequences  of  an  accident  evaluated  previously  in  the  SAR? 

3. 

May  the  proposed  activity  increase  the  probability  of  occurrence  of  a  malfunction  of  equipment  important 
to  safety  evaluated  previously  in  the  SAR? 

4. 

May  the  proposed  activity  increase  the  consequences  of  a  malfunction  of  equipment  important  to  safety 
evaluated  previously  in  the  SAR? 

5. 

May  the  proposed  activity  create  the  possibility  of  an  accident  of  a  different  type  than  any  evaluated 
previously  in  the  SAR? 

6. 

May  the  proposed  activity  create  the  possibility  of  a  malfiinction  of  equipment  important  to  safety  when 
the  malfunction  is  of  a  different  type  than  any  evaluated  previously  in  the  SAR? 

7. 

Does  the  proposed  activity  reduce  the  margin  of  safety  as  defined  in  the  basis  for  any  technical 
specification? 

Is  the  non-safety  related  SSC  used  in  emergency  operating  procedures? 

Could  the  non-safety  related  SSC  prevent  a  safety  related  SSC  from  fulfilling  its  function? 
Could  the  non-safety  related  SSC  cause  a  scram  or  safety  system  actuation? 


A  third  approach  follows  the  NRC’s  recent  recommendations  in  Regulatory  Guide  1.174  for  using  risk  information  in 
evaluating  design  changes  to  a  plant’s  licensing  basis  (NRC,  1998).  The  basic  method  is  as  follows: 

•  Define  the  proposed  change  including  the  aspects  of  the  plant’s  licensing  basis  affected 

•  Perform  engineering  analysis  to  justify  the  proposed  change,  including 

-  evaluation  of  defense-in-depth  and  safety  margins, 

evaluation  of  the  risk  impact  of  the  modification  using  a  suitable  PRA, 

-  comparison  of  the  PRA  result  with  acceptance  guidelines,  and 

integrated  decision  making  to  arrive  at  a  conclusion  about  the  acceptability  of  the  proposed  modification 
after  considering  all  information 
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•  Define  an  implementation  and  monitoring  program  to  ensure  that  there  is  no  adverse  impact  on  safety  once  the 
changes  are  in  place. 

Thus,  an  upgrade  may  be  considered  potentially  safety  significant  if  it  constitutes  an  unreviewed  safety  question,  affects  an 
SSC  that  satisfies  one  of  the  criteria  of  the  Maintenance  Rule,  or  satisfies  the  criteria  of  Reg.  Guide  1.174. 

5.1.2  HFE  Program  Management 

The  planning  phase  contains  one  NUREG-071 1  element,  HFE  Program  Management.  This  review  should  be  conducted,  in 
accordance  with  Element  1  in  Section  2,  for  upgrades  that  affect  the  role  of  personnel  or  the  tasks  by  which  their  role  is 
performed,  and  are  potentially  significant  to  plant  safety.  For  such  upgrades,  the  licensee  should  prepare  an  HFE  program, 
which  is  a  technical  plan  describing  how  HFE  considerations  will  be  addressed  in  designing,  developing,  and  implementing 
the  upgrade.  The  goal  of  this  review  is  to  assess  the  acceptability  of  that  plan. 

Section  2.4  of  NUREG-071 1  has  criteria  for  reviewing  an  HFE  program.  The  following  discusses  topics  that  are  especially 
important  for  reviewing  upgrades: 

•  HFE  program’s  scope 

•  HFE  program’s  goals 

•  HFE  team  and  organization 

•  HFE  process  and  procedures 

•  HFE  issues  tracking 

HFE  Program  Scope 

When  defining  the  program  scope,  the  factors  to  consider  are  the  type  of  upgrade  and  the  characteristics  of  the 
implementation  process. 

Type  of  Upgrade  -  Section  2.4.1  of  NUREG-071 1  specifies  a  broad  range  of  plant  facilities  (Criterion  3)  and  all  HSI 
components  used  for  operations,  accident  management,  maintenance,  test,  inspection,  and  surveillance  (Criterion  4).  For 
an  upgrade,  this  scope  may  be  limited  to  the  HSI  components  included  in  the  upgrade,  and  to  components  of  the  existing 
HSI  that  interact  with  them.  For  example,  if  the  upgraded  controls  and  displays  will  be  used  with  existing  ones,  the 
program  should  assess  their  consistency.  In  addition,  the  scope  should  include  upgrades  to  plant  systems.  For  example,  an 
upgrade  may  entail  modifying  a  system  but  not  the  HSI.  In  this  case,  the  HFE  program  should  ensure  that  the  existing  HSI 
is  compatible  with  the  upgraded  plant  system.  If  the  latter  imposes  new  control  and  display  requirements,  then  the  HFE 
program  should  ensure  that  they  are  identified  and  dealt  with  by  HSI  design,  procedures,  and  training. 

Characteristics  of  the  Implementation  Process  -  The  licensee  may  not  implement  an  entire  upgrade  all  at  once;  instead, 
portions  may  be  installed  in  stages.  For  example,  if  several  systems  are  to  be  upgraded,  the  work  may  extend  over  several 
plant  outages,  with  different  systems  being  implemented  in  each.  In  addition,  the  old  equipment  may  not  be  removed  when 
the  new  equipment  is  installed,  creating  a  situation  in  which  the  composition  of  the  HSI  changes  over  time.  That  is,  the 
composition  that  exists  during  intermediate  periods  differs  from  both  the  initial  design  and  the  final  intended  design.  In 
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addition,  temporary  “fixes”  during  these  intermediate  periods  may  support  the  operation  of  the  HSI  and  plant  systems. 

When  planning  an  HFE  review,  it  is  important  to  address  these  interim  configurations  because  they  may  impose  special 
demands  on  human  performance  that  are  different  from  those  of  the  original  and  the  final  intended  HSI  configuration. 

The  following  provides  four  examples  of  implementing  upgrades,  based  on  the  considerations  discussed.  For  each,  the 
upgrade  may  consist  of  a  small  number  of  HSI  components,  or  the  entire  HSI: 

•  Complete  replacement  -  The  old  HSI  component(s)  are  removed  and  completely  replaced  with  new  ones.  This  is 
accomplished  all  at  one  time,  as  during  a  single  outage.  Personnel  must  adapt  to  a  sudden,  complete  change. 

•  Phased  implementation  -  The  HSI  components  are  gradually  removed  and  replaced  at  intervals  until  all  the  new 
equipment  has  been  installed.  For  example,  one  petrochemical  plant  installed  a  new  CR,  consisting  of  three 
separate  operator  consoles,  in  a  separate  location  near  the  old  CR.  The  transition  was  accomplished  in  three  steps. 
First,  one  of  the  three  consoles  was  installed  in  the  new  CR,  and  the  other  two  new  consoles  in  the  old  CR. 
Operators  in  the  new  CR  coordinated  their  tasks  with  operators  in  the  old  CR  via  telephone.  Operators  controlled 
the  plant  using  both  the  old  and  new  instrumentation.  Second,  all  of  the  new  consoles  were  installed  in  the  new 
CR  and  operations  were  conducted  from  the  new  location.  Third,  after  the  new  CR  had  operated  for  three  months, 
some  advanced  control  capabilities  that  the  old  HSI  design  lacked  were  added  to  the  new  one.  This  completed  the 
gradual  installation  process  (O’Hara,  Stubler,  and  Higgins,  1996).  However,  personnel  had  to  adapt  to  a 
continually  changing  work  environment,  including  temporary  configurations  that  may  differ  from  both  the  original 
and  the  final  configurations. 

•  Dual  installation  with  delayed  switchover  -  New  equipment  may  be  installed  in  the  HSI  while  the  old  HSI 
equipment  is  still  present.  At  some  point,  a  switchover  occurs,  the  new  equipment  is  put  into  service,  and  the  old 
equipment  is  taken  out-of-service.  The  old  equipment  may  be  removed,  or  disconnected  and  tagged  out-of-service 
(e.g.,  abandoned  in  place).  This  approach  may  be  desirable  in  requiring  a  new  system  to  pass  a  trial  period  before 
plant  personnel  are  confident  in  its  operation.  For  example,  Higgins  and  Pope  (1996)  describe  the  switchover  from 
an  old  set  of  display  devices  to  a  newly  installed  cathode  ray  tube  (CRT)-based  display  system  in  the  CR  of  a  U.S. 
government  facility.  Before  the  switchover,  the  new  display  system  was  driven  by  a  computer-based  simulator. 
This  gave  the  operators  an  opportunity  to  learn  about,  and  use,  the  new  display  system  in  their  spare  time,  while 
the  facility  was  being  operated  from  the  old  HSI  components.  At  some  point,  the  new  display  system  was 
disconnected  from  the  simulation,  and  linked  directly  to  the  plant.  Here,  personnel  must  adapt  to  a  continually 
changing  work  environment.  They  must  identify  which  HSI  components  are  functional,  including  display  devices 
that  may  be  driven  by  a  simulation,  rather  than  by  the  actual  plant. 

•  Dual  installation  with  no  switchover  -  In  some  cases,  the  new  equipment  is  redundant  with  the  existing  equipment; 
accordingly,  both  old  and  new  equipment  are  operational  and  personnel  may  use  either  one.  This  approach  may  be 
more  common  for  display  systems  (e.g.,  when  a  computer-based  display  system  is  installed  to  augment  the  existing 
displays).  Dual  installation  may  be  less  common  for  control  system  upgrades  because  of  practical  considerations 
involving  redundant  controllers  (e.g.,  preventing  conflicts  due  to  simultaneous  use).  Dual  installation  can  allow 
operators  to  revert  to  the  old  equipment  if  they  encounter  problems  with  the  new  equipment.  In  this  case, 
personnel  must  adapt  to  an  HSI  that  is  somewhat  more  complex  because  it  has  different  HSI  components  that 
perform  similar  functions  but  may  have  different  methods  of  operation. 

The  demands  imposed  on  personnel  for  adapting  to  the  changes  in  the  HSI  are  different  for  each  of  these  examples,  and 
should  be  considered  when  reviewing  an  HFE  upgrade. 
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HFE  Program  Goals 

Criterion  1  of  Section  2.4.1  of  NUREG-071 1  describes  the  general  goals  that  should  be  reflected  in  an  HFE  program.  For 
an  upgrade,  the  introduction  of  changes  to  the  plant  will  require  modifying  and  adapting  existing  work  processes  and 
practices.  The  way  in  which  upgrades  are  implemented  may  impose  special  demands  on  personnel,  and  the  HFE  program 
goals  should  consider  such  possible  effects.  The  transition  from  the  existing  plant  configuration  to  the  upgraded 
configuration  should  be  planned  so  that  minimal  demands  are  imposed  on  personnel  adapting  to  the  change.  The 
considerations  should  include  the  following; 

•  Planning  the  installation  process  to  minimize  disruptions  to  ongoing  activities, 

•  Coordinating  training  and  procedure  modification  with  the  implementation  of  the  upgrade  to  ensure  integration, 
and 

•  Conducting  training  to  maximize  operators’  knowledge  and  skill  with  the  new  design  before  using  it. 

HFE  Team  and  Organization 

Section  2.4.2  of  NUREG-071 1  gives  criteria  for  the  responsibility,  authority,  composition,  and  staffmg  of  the  licensee’s 
HFE  team.  In  an  upgrade,  personnel  who  will  be  the  ultimate  users  of  the  upgrade  can  be  valuable  sources  of  domain- 
specific  information  that  should  be  considered  when  designing,  developing,  and  evaluating  upgrades.  Their  involvement 
should  ensure  that  the  following  points  are  considered; 

•  Knowledge  of  task  demands  and  constraints  of  the  work  environment 

•  Knowledge  of  work  processes 

•  Knowledge  of  organizational  goals  that  affect  the  use  of  the  upgrade. 

The  involvement  of  workers  in  the  development  process  also  can  also  help  ensure  that  the  final  design  of  the  upgrade  is 
consistent  with  organizational  concerns  such  as  reliability,  maintainability,  and  supportability  (Medsker  and  Campion, 

1997;  Price,  1990).  For  example,  Roth  and  O’Hara  (1998)  reviewed  a  new  computer-based  display  system  in  a  NPP.  They 
found  that  although  operations  personnel  had  been  actively  involved  in  developing  the  system,  the  plant’s  management 
believed  that  their  greater  involvement  would  have  been  beneficial.  (Additional  discussion  of  this  topic  is  given  in  Section 
5.5,  Organizational  Considerations  for  Designing  and  Implementing  HSI  Upgrades.) 

HFE  Process  and  Procedures 


Section  2.4.3  of  NUREG-071 1  provides  a  criterion  for  ensuring  that  vendors  and  subcontractors  address  HFE  in  developing 
upgrades.  The  HFE  program  should  ensure  that  the  HFE  requirements  are  specified  in  agreements  with  the  vendors  and 
contractors.  This  criterion  is  especially  important  for  upgrades  because  add-ons  or  modifications  to  the  HSI  and  plant 
systems  are  often  purchased  as  modular  units  designed  to  the  standard  specifications  of  the  vendor,  rather  than  to  a 
particular  plant’s  specifications. 
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HFE  Issues  Trackina 

Section  2.4.4  of  NUREG-071 1  provides  criteria  for  ensuring  that  the  licensee  has  established  a  means  for  documenting  and 
tracking  HFE  issues  that  arise  during  the  design  process.  It  states  that  HFE  issues  should  be  tracked  from  identification 
until  elimination  or  reduction  to  an  acceptable  level.  However,  some  HFE  issues  may  not  be  detected  during  HFE 
verification  and  validation,  but  may  become  apparent  during  plant  operation.  For  example,  when  setting  up  a  computer- 
based  display,  management  decided  that  it  would  be  beneficial  to  establish  a  systematic  formal  approach  for  recording 
problems  and  suggestions  arising  during  final  adjustments  of  the  system  (Roth  and  O’Hara,  1998).  HFE  considerations 
identified  in  this  way  should  also  be  recorded  in  an  HFE  issues  tracking  system;  they  should  be  documented  and  tracked  as 
described  in  Section  2.4.4  of  NUREG-071 1. 

5.2  Analysis  Phase 

The  analysis  phase  identifies  the  requirements  for  new  designs.  NUREG-071 1  recommends  a  top-down  approach  which 
includes  identifying  functions,  tasks,  staffing,  and  human-reliability  considerations.  In  developing  an  HSI  upgrade,  the 
focus  should  be  on  characteristics  of  the  new  design  that  differ  from  the  predecessor  design,  and  their  potential  effects  on 
people’s  performance.  The  extent  of  these  analyses  should  be  commensurate  with  the  scope  of  the  upgrade  and  the  potential 
consequences  to  performance  and  plant  safety.  This  section  describes  human  performance  considerations  and  methods  for 
analyzing  upgrades,  and  addresses  the  following  considerations: 

•  Functional  requirements 

•  Task  requirements 

•  Cognitive  task  analysis 

•  Crew  performance  and  staffing 

•  Human  reliability  and  design  bases 

5.2.1  Functional  Requirements  Analysis  and  Function  Allocation 

Element  3  in  Section  4  of  NUREG-071 1  describes  two  analyses:  a  functional  requirements  analysis  and  a  function 
allocation  analysis.  The  purpose  of  the  former  is  to  ensure  that  the  plant’s  functional  requirements  have  been  defined. 
Functional  requirements  analyses  should  be  reviewed,  in  accordance  with  Section  4  of  NUREG-071 1,  for  all  upgrades  that 
are  safety  significant,  that  affect  the  role  of  personnel,  and  are  likely  to  change  existing  functions  that  are  important  to 
safety,  introduce  new  functions,  or  involve  requirements  that  are  not  clearly  defined  and  may  be  important  to  safety.  The 
scope  of  the  analyses  may  be  restricted  to  functions  related  to  the  upgrade.  The  purpose  of  frinction  allocation  analysis  is  to 
ensure  that  functions  are  allocated  to  take  advantage  of  human  strengths,  and  to  avoid  allocating  them  to  personnel  if  they 
would  be  negatively  affected  by  human  limitations.  Function  allocation  analyses  should  be  reviewed,  in  accordance  with 
Section  4  of  NUREG-071 1,  for  all  upgrades  that  are  safety  significant,  that  affect  the  role  of  personnel,  and  are  likely  to 
change  the  allocation  between  personnel  and  plant  systems  of  functions  important  to  safety.  The  scope  of  the  analyses  may 
be  restricted  to  functions  related  to  the  upgrade. 

Changes  in  plant  systems  may  change  the  way  existing  functions  are  carried  out.  For  example,  hydrogen  ignitors  could  be 
an  alternative  way  to  eliminate  this  gas  from  the  containment.  As  another  example,  some  upgrades  of  digital  feedwater 
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systems  for  PWRs  have  an  automatic  switch-over  from  the  auxiliary  to  the  main  feedwater  systems  during  startup.  This 
provides  continuous  automatic  control  of  feedwater  over  a  broad  range  of  reactor  power  conditions,  whereas  the  original 
configuration  may  have  consisted  of  separate  control  systems  for  the  auxiliary  and  main  feedwater  systems.  Thus,  the 
function  of  switching  from  one  system  to  another  still  exists  with  the  new  system,  but  it  is  performed  differently.  As  a  third 
example,  an  upgrade  may  change  the  behavior  of  a  system  performing  a  frmction,  so  that  it  may  be  performed  differently 
(e.g.,  when  digital  control  systems  are  installed,  the  systems  often  respond  more  quickly  to  changes  in  control  setpoints  and 
exhibit  less  drift  from  the  setpoint  value). 

These  changes  should  be  identified  in  accordance  with  NUREG-07i  land  NUREG-0700,  and  indications  that  the  function  is 
available,  operating,  and  achieving  its  purpose  should  be  described. 

Plant  upgrades  also  may  affect  the  allocation  of  new  and  existing  functions  to  personnel.  Automation  of  plant  systems  has 
resulted  in  a  trend  away  from  manual  operation,  and  concomitantly,  in  more  complex  automated  control  modes. 
Accordingly,  the  operator’s  role  is  shifting  away  from  directly  controlling  plant  systems  and  toward  monitoring  automated 
systems  and  intervening  when  they  are  not  functioning  properly  (O’Hara,  Stubler,  and  Higgins,  1996).  Analyses  should 
ensure  that  the  ftmctions  allocated  to  personnel  is  within  their  capabilities.  An  important  consideration  is  the  ability  of 
personnel  to  understand  the  structure  and  function  of  these  automated  systems,  so  they  can  detect  and  respond  to 
malfunctions.  This  understanding  is  affected  both  by  the  design  of  these  systems  and  the  characteristics  of  the  HSl,  which 
conveys  the  information. 

5.2.2  Task  Analysis 

Task  analysis,  as  described  by  Element  4  in  Section  5  of  NUREG-071 1,  is  a  detailed  evaluation  of  the  tasks  that  must  be 
performed  to  accomplish  the  allocated  functions.  It  establishes  the  performance  requirements  for  personnel,  and  provides  a 
basis  for  HSI  design  requirements.  Task  analyses  should  ensure  that  tasks  resulting  from  upgrades  are  within  the 
capabilities  of  the  affected  personnel.  Task  analysis  should  be  reviewed,  in  accordance  with  Section  5  of  NUREG-071 1,  for 
all  upgrades  that  are  safety  significant,  that  affect  the  role  of  personnel,  and  are  likely  to  affect  tasks  previously  identified  as 
risk  important,  cause  existing  ones  to  become  risk  important,  or  create  new  risk-important  tasks.  The  scope  of  the  analyses 
should  include  tasks  involving  the  upgrade  and  its  interactions  with  the  rest  of  the  plant.  For  maintenance,  test,  inspection, 
and  surveillance,  attention  should  be  given  to  new  risk-important  tasks  and  those  supported  by  new  technologies  (e.g.,  new 
capabilities  for  on-line  maintenance). 

As  described  in  Section  5.2.1,  upgrades  to  plant  systems  or  the  HSI  may  change  the  functions  and  hence  the  tasks 
performed  by  personnel.  For  example,  introducing  a  new  automated  system  may  create  a  new  function  in  which  the 
operator  must  monitor  the  system  and  intervene  if  it  malfunctions.  Tasks  that  result  from  the  new  or  changed  personnel 
functions  should  be  the  starting  point  for  the  task  analysis.  Cognitive  requirements  for  these  tasks  may  be  described  in 
terms  of  the  generic  cognitive  tasks  (i.e.,  monitoring  and  detection,  situation  assessment,  response  planning,  and  response 
implementation)  described  in  Section  4.  In  addition,  people’s  tasks  may  be  changed  by  the  upgrade  even  if  their  functions 
are  not.  That  is,  the  same  functions  may  be  performed  in  different  ways  when  a  new  HSI  component  is  used;  therefore,  the 
task  analysis  should  also  consider  such  changes. 

The  following  describes  the  specific  characteristics  of  tasks  that  may  be  affected  by  upgrades  and,  therefore,  should  be 
addressed  by  the  task  analysis.  First,  the  task’s  characteristics  are  described  at  the  level  of  the  individual.  Next,  those  that 
affect  the  crew’s  performance  are  discussed.  Finally,  we  discuss  task  analysis  methods  specifically  focused  on  assessing 
cognitive  demands.  These  methods  may  be  useful  for  analyzing  knowledge  and  cognitive  skills  associated  with  using  an 
upgrade. 
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Chan£es  in  Individual  Crew  Members’  Tasks 

Human  performance  is  affected  by  the  limitations  of  working  memory  and  attention  by  restricting  the  amount  of 
information  that  can  be  processed  at  any  given  time.  Experienced  users  compensate  by  employing  specific  cognitive  skills 
and  strategies  which  are  tailored  to  the  task  domain,  and  tailored  after  the  types  of  information  handled,  and  its  sources. 
When  systems  develop  over  a  long  period,  as  with  conventional  HSI  designs,  a  natural  interactive  process  may  develop 
between  users  and  designers  (Rasmussen  and  Goodstein,  1988).  For  example,  users  may  develop  skills  for  coping  with  the 
initial  design  of  the  system,  and  they  may  become  part  of  a  formal  or  informal  training  program.  Designers  recognize  these 
skills  and  incorporate  features  that  utilize  these  skills  in  the  next  version  of  the  product.  The  users  then  refine  and  adapt 
these  skills  to  the  new  design,  which  designers  recognize  and  accommodate  in  subsequent  designs.  Over  time,  these  design 
characteristics  become  part  of  the  craft  of  designing  the  product  or  system,  and  the  explicit  rationale  for  why  certain  HSI 
components  have  particular  characteristics  may  be  lost.  Rasmussen  and  Goodstein  (1988)  describe  this  phase  as  stable 
periods  characterized  by  “horizontal”  design  improvements.  New  components  or  user  requirements  are  incorporated  within 
an  established  framework  of  which  the  conceptual  basis  and  “...original  reasons  can  no  longer  be  explicitly  formulated” 
(Rasmussen  and  Goodstein,  1988,  p.  179). 

However,  when  new  HSI  technologies  are  introduced,  such  as  computer-based  display  systems,  the  features  of  the  HSI  may 
change  radically  from  the  traditional  design.  That  is,  “...the  introduction  of  new  technology,  as  illustrated  by  the  on-march 
of  advanced  information  systems  destroys  the  blaitket  of  established  know-how  and  practices”  (Rasmussen  and  Goodstein, 
1988,  p.  179).  Consequently,  the  strategies  developed  by  experienced  users,  based  on  the  old  design,  may  no  longer  be 
effective. 

An  early  example  occurred  in  the  nuclear  industry  when  tile-based  annunicators  were  replaced  by  alarm-list  displays.  The 
designers  considered  the  alarm  system  from  a  fairly  narrow  perspective,  focusing  on  its  function  as  an  automated 
monitoring  system  which  alerted  operators  to  off-normal  parameters.  It  was  subsequently  discovered  that  operators  use  the 
alarm  system’s  information  in  many  ways  which  may  not  be  obviously  based  on  the  original  intent  of  the  design.  For 
example,  the  annunciator  tiles  were  used  as  an  overview  display  to  support  response  planning  (O’Hara  and  Brown,  1994; 
O’Hara,  Brown,  Higgins,  and  Stabler,  1994;  Pew,  et  al.,  1981).  Operators  learned  the  locations  of  annunciator  tiles,  and 
could  quickly  assess  the  plant’s  overall  condition  from  the  pattern  of  illuminated  ones.  The  presentation  of  alarm 
information  in  the  alarm-list  displays  lacked  spatial  dedication  and  permanence  (i.e.,  the  alarm  messages  scrolled  off  the 
display  page).  As  a  result,  the  workload  associated  with  these  list  displays  increased  greatly  and  they  were  found  to  be 
inadequate  replacements  for  the  tiles. 

Another  example  is  related  to  the  current  trend  in  the  design  of  new  control  rooms  toward  compact,  computer-based  control 
panels.  NPP  CRs  have  traditionally  featured  broad  panels  of  individual,  spatially  dedicated  controls  and  displays.  While 
this  configuration  posed  many  demands  on  operators  (e.g.,  in  integrating  information  across  broadly  distributed 
instruments),  it  also  had  benefits  for  coordinating  crew  activities.  Operators  shared  the  same  view  of  plant  instruments  and 
could  monitor  each  other’s  performance  and  detect  errors  fairly  easily.  Some  of  these  benefits  may  be  lost  in  compact  CRs 
where  operators  have  separate  consoles  with  individual  views  of  the  plant  that  cannot  be  readily  shared  with  other  operators 
(Stabler  and  O’Hara,  1996b). 

Thus,  while  traditional  HSI  designs  may  have  many  limitations,  they  may  support  skilled  performance  in  ways  that  are  not 
immediately  obvious,  and  also  support  functions  or  tasks  not  explicitly  stated  in  the  design  requirements.  These  functions 
and  tasks  may  have  evolved  over  time  from  user’s  strategies  for  adapting  to  the  specific  features  of  an  established  design. 
When  introducing  new  technologies,  it  is  important  to  investigate  the  skills  used  previously  to  discover  these  functions, 
tasks,  and  strategies;  this  may  require  techniques  specifically  focused  on  the  skills  and  strategies  that  define  user  expertise 
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in  the  old  configuration.  (See  discussion  of  cognitive  task  analysis  below.)  If  these  user’s  needs  are  not  identified  during 
task  analysis  and  considered  in  the  design  activity,  the  modification  may  fail  as  a  replacement  for  the  current  design. 

When  analyzing  tasks,  attention  should  be  given  to  the  strategies  used  for  performing  them,  the  information  obtained,  and 
the  HSI  characteristics  that  support  these  strategies.  The  following  describes  three  types  of  strategies:  task  automaticity, 
information  chunking,  and  HSI  tailoring. 

Performing  Tasks  Without  Highly  Focused  Attention  -  Automaticity,  performing  actions  without  committing  a  high  degree 
of  conscious  mental  resources,  allows  personnel  to  free-up  cognitive  resources  for  other  tasks.  Automaticity  is  a  mark  of 
proficient  users.  However,  it  may  not  be  achieved  if  the  HSI  does  not  minimize  demands  on  the  user.  Task  analysis  should 
address  automaticity  in  two  ways.  It  should  identify  (1)  actions  that  would  benefit  from  automaticity,  and  (2)  actions  for 
which  automaticity  should  be  discouraged. 

According  to  Schlager  et  al.  (1989),  actions  may  be  candidates  for  automaticity  if  they  : 

•  Are  associated  with  a  discemable  event, 

•  Are  performed  the  same  way  each  time, 

•  Do  not  require  a  high  degree  of  attention  (e.g.,  do  not  involve  a  high  degree  of  focused  attention  in  planning  or 
executing  a  response),  and 

•  Are  not  interrupted  by  unique  actions. 

The  discemable  event  is  a  stimulus  that  triggers  the  response,  which  may  consist  of  a  single  action,  or  a  pattern  of  actions  in 
a  specific  sequence.  Actions  performed  the  same  way  every  time  a  stimulus  situation  occurs  are  called  consistent  task 
components  because  the  stimulus  and  response  are  consistently  mapped  (Scheider,  1985).  Variable  task  components  are 
actions  that  are  not  carried  out  identically  each  time  and,  consequently,  are  sensitive  to  the  availability  of  cognitive 
resources.  Consistent  task  components  show  large  improvements  in  performance  with  practice,  while  variable  task 
components  show  little  or  none  (Schneider,  1985). 

Some  activities  that  satisfy  these  criteria  may  include  routine  control  actions  and  interface  management  (e.g.,  accessing 
displays  from  a  display  system).  Personnel  normally  are  expected  to  do  them  without  focusing  on  the  details  of  their 
execution.  Candidates  for  automaticity  should  be  identified  during  task  analysis  and  included  in  designing  the  HSI  and 
developing  training  programs  (Mumaw  and  Gabrys,  1996;  Schlager  et  al.,  1989). 

Automaticity  should  be  discouraged  when  accidental  or  incorrect  operation  of  a  system  may  have  significant  consequences 
for  plant  safety.  As  described  in  Section  4.3.4,  slips  are  errors  involving  the  incorrect  execution  of  an  intention.  They  are 
the  undesirable  consequence  of  automaticity  because  they  occur  when  a  pattern  of  actions  intended  to  accomplish  a 
particular  intention  gets  waylaid  en  route  to  execution  (Norman,  1983;  Norman,  1988).  A  review  of  soft  controls  used  in 
hybrid  HSIs  found  that  computer-based  user  interfaces  are  especially  prone  to  some  types  of  slips  (Stubler,  O’Hara,  and 
Kramer,  2000);  users  may  operate  the  wrong  control,  or  carry  out  the  wrong  action  on  the  right  control.  A  careful  task 
analysis  should  identify  tasks  for  which  automaticity  is  not  desirable,  including  tasks  for  which  the  consequences  of  errors 
are  high,  such  as  those  deemed  risk  important  by  an  HRA,  and  others  that  affect  the  operation  of  safety  systems.  Some 
examples  include  initiating  or  terminating  operation  of  a  plant  protection  system,  entry  of  data  that  would  affect  the 
operation  of  a  plant  protection  system,  and  any  other  control  actions,  which  if  performed  incorrectly,  may  initiate  a 
transient.  They  should  be  addressed  in  the  HSI  design  process  by  applying  approaches  to  prevent  errors. 
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Information  C/2w«A:/>7g  -  Experienced  users  sometimes  handle  large  volumes  of  data  by  using  the  cognitive  skill  of 
information  chunking,  organizing  individual  bits  of  information  into  conceptually  related  groups.  Schlager  et  al.,  (1989) 
describe  strategies  used  by  experienced  air  traffic  controllers  allowing  them  to  handle  large  volumes  of  traffic  in  the  air 
sector  they  control.  Thus,  the  pattern  of  many  aircraft  trajectories  may  be  organized  into  more  manageable  groups  by 
considering  those  traveling  inbound  and  those  traveling  outbound,  which  require  different  responses  from  the  controller. 
These  groups  may  be  further  organized  into  subgroups  based  on  distance,  elevation,  and  aircraft  type. 

Users  often  rely  on  the  characteristics  of  the  HSI  to  formulate  and  use  information-chunking  strategies.  The  air  traffic 
controllers  may  rely  on  display  icons  to  distinguish  between  inbound  and  outbound  aircraft,  or  they  may  access  displays  that 
present  them  separately.  For  NPP  operations,  operators  may  rely  on  display  characteristics,  such  as  spatial  arrangement  and 
visual  coding  (e.g.  color,  flashing).  Different  strategies  may  be  employed  depending  upon  how  the  information  is  depicted 
(e.g.,  alphabetically,  chronologically,  or  by  function),  and  users  develop  skills  based  on  such  presentations. 

HSI  Tailoring -WYE  design  principles  dictate  that  the  HSI  should  be  designed  to  be  compatible  with  human  capabilities 
and  limitations.  However,  matching  the  HSI  to  human  capabilities  may  continue  after  it  is  installed.  Users  may  tailor  the 
HSI  to  their  specific  tasks  or  personal  characteristics.  This  may  be  done  by  augmenting  the  information  provided  by  the 
user  interface  with  additional  data  or  by  manipulating  HSI  features  to  draw  attention  to  particular  information.  One 
strategy  is  to  use  reminders  which  help  the  user  maintain  important  information  in  memory,  or  assist  in  accessing  it  later. 
Vicente,  Mumaw,  and  Roth  (1997)  describe  several  methods  used  by  CR  operators.  In  one  case,  operators  wanted  to  focus 
on  monitoring  a  particular  variable  shown  on  a  strip-chart  recorder.  Since  there  were  no  special  mechanisms  for  drawing 
attention  to  the  recorder  or  for  reminding  them  that  the  variable  should  be  watched  more  often,  the  operators  developed  a 
practice  of  sliding  the  recorder  forward  from  its  normal  position  in  the  control  panel;  its  forward  projection  accomplished 
these  aims. 

Also,  Vicente  et  al.  noted  that  operators  attached  small  notes  to  control  and  display  devices  on  the  control  panel  to  draw 
attention  to  the  device  or  to  record  important  information,  such  as  the  previous  values  of  a  closely  watched  variable  or  a 
reference  value;  this  reminded  them  to  act  when  a  particular  value  is  reached.  (While  this  may  not  be  a  recommended 
practice,  it  is  given  as  an  example  of  users  modifying  the  HSI  to  cope  with  demands  unrecognized  when  the  HSI  was 
designed.) 

Another  example  by  Schlager  et  al.  (1989)  describes  how  air  traffic  controllers  manipulate  their  flight  strips.  Flight  strips 
are  labels  describing  aircraft  in  a  particular  air  sector.  They  are  normally  attached  to  plastic  strips  and  stacked  in  racks  at  a 
controller’s  console  so  they  may  be  easily  viewed.  One  strategy  controllers  use  to  remind  them  of  aircraft  requiring  special 
attention  is  to  “cock”  the  flight  strip  for  that  aircraft.  Alternatively,  controllers  may  mark  the  strip  with  large  dark  letters  to 
draw  attention  to  particular  details. 

The  user’s  modifications  of  the  HSI  are  important  for  two  reasons.  First,  the  fact  that  the  operators  find  the  modifications 
necessary  suggests  that  HSI  designs  may  not  fully  meet  the  task  requirements  (i.e.,  the  original  design  has  limitations). 

When  developing  an  upgrade  or  replacement  system,  the  users’  modifications  should  be  studied  through  task  analyses  to 
determine  how  and  when  they  are  used  to  accomplish  work  goals.  The  functions  and  tasks  performed  using  these 
modifications  should  be  included  in  the  design  requirements  for  the  upgrade.  Second,  these  modifications  illustrate 
practical  solutions  for  particular  needs.  If  this  function  is  needed  in  the  new  design,  then  the  design  should  include  some 
mechanism  allowing  the  operators  to  accomplish  the  same  result.  Ideally,  that  new  mechanism  should  be  consistent  with 
the  method  used  in  the  old  design,  so  operators  do  not  have  to  learn  new  skills. 

Using  the  flight  strips  as  an  example,  an  upgrade  could  replace  the  plastic  flight  strips  with  labels  on  a  CRT.  The  air  traffic 
controllers’  strategies  of  cocking  and  marking  the  plastic  strips  should  be  analyzed  to  determine  when  they  are  used  and 
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what  information  they  impart.  Features  should  be  included  in  the  new  design  that  convey  the  same  information  provided  by 
cocking  and  marking  the  strips.  Training  requirements  may  be  reduced  to  the  degree  that  the  new  methods  are  similar  to 
those  of  the  original  design. 

It  is  important  to  ensure  that  skills  and  strategies  developed  by  users  for  the  original  design  are  either  supported  by  the  new 
design  or  no  longer  needed  due  to  improved  work  methods.  Situations  should  be  avoided  in  which  the  new  work  methods 
are  not  a  significant  improvement,  and  the  old  strategies  are  not  applicable  to  the  new  design.  In  such  cases,  the  end  result 
may  be  that  workload  is  increased  rather  than  decreased  by  the  new  design. 

Changes  in  Crew  Tasks 

When  analyzing  a  task,  it  is  important  to  consider  the  distribution  of  demands  among  personnel.  Additional 
communications  and  coordination  may  be  necessary  to  facilitate  new  demands  associated  with  monitoring  and  detection, 
situation  assessment,  response  planning,  and  response  implementation.  Section  4.4.2  described  the  crew’s  performance  and 
team  skills  within  the  context  of  these  four  generic  cognitive  tasks.  Important  factors  included  a  shared  context  (e.g., 
common  understanding  of  the  plant’s  state)  between  the  speaker  and  the  listener  to  correctly  interpret  intended  meanings, 
the  ability  of  decision  makers  to  communicate  the  intermediate  results  of  assessment  and  planning,  and  the  ability  of  other 
personnel  to  gather  and  evaluate  information  in  support  of  the  primary  decision  maker.  The  characteristics  of  the  work 
environment  (e.g.,  horizon  of  observation,  openness  of  tools,  and  of  interactions)  can  help  persoimel  in  communicating  and 
in  coordinating  work.  The  task  analysis  should  identify  any  changes  in  such  demands  and  also  identify  design 
characteristics  that  support  them  in  the  current  work  environment  but  which  may  not  exist  in  the  same  form  in  the  upgraded 
HSI.  These  factors  should  be  highlighted  to  support  evaluations  of  staffing,  HSI  design,  procedures,  and  training  in  later 
stages  of  the  HFE  review. 

Methods  of  Cognitive  Task  Analysis 

Conventional  task  analyses  tend  to  focus  on  explicit  and  objective  knowledge,  such  as  actions  that  can  be  easily  observed  or 
described  in  procedures.  While  such  analyses  are  necessary  for  defining  human  performance  for  routine  work,  they  may  be 
insufficient  for  situations  in  which  high  demands  are  imposed  on  cognitive  capabilities  (Klein,  Calderwood,  and 
MacGregor,  1989).  These  analyses  may  need  to  be  supplemented  with  techniques  specifically  focusing  on  how  personnel 
access  and  process  information  in  the  work  environment.  Cognitive  Task  Analysis  (CTA)  methods  have  grown  out  of  the 
need  to  understand  the  information  processing  demands  imposed  by  working  in  complex  human-machine  systems.  The 
methods  can  identify  the  range  of  problems  that  personnel  encounter  in  a  domain,  define  the  dimensions  of  task  complexity, 
and  define  the  requirements  for  information  and  problem  solving.  CTA  is  similar  to  standard  task  analysis  in  its  concern 
for  inputs,  processing,  and  outputs.  However,  its  focus  is  on  cognitive  operations  undertaken  to  acquire  and  transform  the 
information  for  executing  an  action  or  making  a  decision  (Sanquist,  Lee,  and  McCallum,  1995). 

CTA  may  be  applied  to  standard  function  and  task  analyses  to  examine  them  in  greater  detail.  For  example,  for  functions 
and  tasks  particularly  important  to  the  role  of  the  operator,  CTA  can  identify  the  individual  cognitive  transactions  necessary 
to  extract  and  process  specific  data.  It  can  be  useful  for  examining  the  role  of  different  technologies  in  operators’  cognitive 
tasks.  For  example,  it  may  be  used  to  define  the  effects  of  automation  on  a  person’s  decision-making  role  or  the  effects  of 
information  technologies  in  supporting  cognitive  functions  associated  with  specific  tasks.  The  findings  can  guide  the 
development  of  design  requirements  for  upgrades,  or  establish  requirements  for  training. 

Roth  and  Mumaw  (1995)  identify  three  categories  of  CTA:  function-based,  empirical,  and  cognitive  simulation.  They  note 
that  most  CTA  methods  incorporate  features  from  more  than  one  of  them.  Table  5.2  lists  some  important  characteristics  of 
these  approaches. 
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Table  5.2  Comparison  of  Three  Categories  of  Cognitive  Task  Analysis  (CTA) 


CTA 

Category 

System  Being  Studied 

Cognitive  Factors 

Analyst’s  Activities 

Analyst’s 

Expertise 

Empirical 

Existing  (actual  or 
simulator) 

Specific  knowledge  and 
skills  used  in  actual 
practice 

Factors  that  distinguish 
levels  of  expertise 

Conducting 

observations,  interviews 
and  walk-throughs  with 
subject-matter  experts 

Cognition 

Function- 

based 

Existing  or  future 

Information 

requirements  for  specific 
decisions 

Analyzing  information 
requirements 

Systems 

engineering 

Cognitive 

Simulation 

Existing  or  future 

Human  response  time 
and  errors 

Consequences  of  errors 

Eliciting  expert 
knowledge  and 
representing  it  in 
software 

Cognition  and 

software 

development 

Empirical  CTA  -  These  approaches  rely  on  empirical  techniques  to  analyze  how  people  perform  tasks  in  either  real  or 
simulated  environments.  Their  focus  is  to  identify  the  knowledge  and  skills  used  by  practitioners  in  a  specific  domain 
(Klein,  Calderwood,  and  MacGregor,  1989;  Roth  and  Woods,  1988).  They  are  particularly  well  suited  for  identifying 
factors  that  account  for  performance  differences  between  experts  and  less-experienced  practitioners.  Empirical  CTA 
techniques  require  the  analyst  to  have  a  background  in  HFE  with  an  understanding  of  cognition.  These  techniques  may 
employ  a  variety  of  collection  methods  to  obtain  data  about  the  skills  and  strategies  used  by  working  personnel.  For 
example,  walk-through  exercises  may  be  used  in  which  personnel  demonstrate  their  skills  while  using  particular  equipment. 
Carefully  crafted  probing  questions  may  be  used  to  focus  on  problem-solving  strategies. 

Empirical  CTA  approaches  are  particularly  appropriate  for  developing  training  programs  for  existing  systems  because  they 
encompass  the  address  dimensions  of  expertise  (e.g.,  special  knowledge  and  skills)  required  for  the  current  design  (Roth 
and  Mumaw,  1995),  including  the  user’s  strategies,  such  as  automaticity,  information  chunking,  and  HSI  tailoring.  Also,  in 
studying  performance  under  defined  event  scenarios,  the  frequency  of  use  of  specific  strategies  and  skills  can  be  counted. 
These  counts,  together  with  users’  assessments  of  the  task’s  difficulty,  may  determine  what  should  be  addressed  in  the 
training  program. 

Empirical  CTA  approaches  also  can  be  used  in  developing  new  HSI  designs  and  upgrades.  Frequency  counts  of  various 
activities  may  serve  as  crude  measures  of  workload  when  evaluating  design  alternatives.  For  example,  an  analysis  may 
show  that  certain  mental-processing  operations  can  be  eliminated  when  a  decision-support  aid  is  used  (Sanquist,  Lee,  and 
McCallum,  1995).  A  CTA  may  reveal  the  skills  and  strategies  personnel  use  and  so  identify  the  design  characteristics  of 
the  HSI  to  support  them.  This  analysis  also  may  determine  the  relevance  of  these  design  characteristics  and  the  associated 
user  strategies  for  the  new  design.  Empirical  CTA  approaches  can  assess  the  consistency  of  the  HSI.  If  they  indicate 
difficulties  in  using  multiple  HSI  components  for  a  particular  task,  this  may  suggest  that  the  interaction  methods  across  the 
HSI  are  inconsistent. 
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The  Critical  Decision  Method  (CDM)  is  a  retrospective  interview  technique  for  studying  the  general  knowledge,  specific 
information,  and  reasoning  processes  used  by  personnel  in  environments  characterized  by  high  time  pressure,  high 
information  content,  and  changing  conditions  (Klein,  Calderwood,  and  MacGregor,  1989).  In  such  environments, 
personnel  often  resort  to  recognition-primed  decision  making.  That  is,  they  select  actions  based  on  recognizing  critical 
information  and  on  prior  knowledge  rather  than  by  engaging  in  deliberative  evaluations  of  all  possible  courses  of  action. 

The  CDM  focuses  on  three  aspects  of  expertise:  explicit  and  objective  knowledge,  tacit  knowledge,  and  perceptual 
knowledge  (perceptual-motor  feel).  Explicit  and  objective  knowledge  is  typically  the  focus  of  a  traditional  task  analysis. 
The  other  two  aspects  represent  categories  of  knowledge  that  are  critical  to  skilled  behavior  but  resistant  to  being  articulated 
(e.g.,  the  ability  of  experts  to  judge  that  a  situation  is  familiar,  without  first  analyzing  it  in  detail).  The  CDM  focuses  on 
actual  incidents  considered  non-routine  in  which  expertise  is  most  important,  and  cognitive  skills  can  be  most  readily 
examined. 

The  method  consists  of  five  steps:  (1)  selecting  the  incident  to  be  explored,  (2)  obtaining  an  unstructured  account  of  it  based 
on  the  SME’s  recollection  and  verbal  description,  (3)  constructing  a  time  line  for  the  incident,  (4)  identifying  points  at 
which  important  decisions  were  made,  and  (5)  probing  the  decision  points.  Probing  is  the  step  in  which  details  are  obtained 
about  cognitive  skills  and  knowledge.  A  standardized  set  of  probes  (focused  questions)  is  used  to  explore  such  topics  as 
cues  used  by  the  SME  to  assess  the  situation,  prior  knowledge  and  experience  used,  goals  considered  in  planning  responses, 
and  options  that  existed  at  the  decision  points. 

One  output  of  this  method  is  a  descriptive  decision  model,  i.e.,  a  model  of  important  decisions  made  by  the  SMEs  that 
identifies  specific  decisions  and  their  characteristics.  A  second  output  is  a  critical  cue  inventory,  a  categorization  of  all 
informational  and  perceptual  cues  used  in  assessing  a  situation  or  considering  options.  Together  they  provide  an 
understanding  of  user  strategies  for  adapting  to  the  current  design  and  HSI  characteristics  that  support  these  strategies. 

They  may  be  used  in  establishing  design  requirements  for  an  HSI  upgrade  or  for  training  requirements.  A  third  output  is 
the  identification  of  non-routine  incidents.  Because  these  are  actual  incidents  that  challenge  the  HSI  design  and  personnel 
expertise,  they  form  a  basis  for  developing  HFE  validation  scenarios. 

Function-Based  CTA  -  Function-based  CTA  relies  on  a  formal  functional  analysis  of  the  applicable  domain  that  is  usually 
based  on  some  form  of  goals-means  functional  decomposition  (e.g.,  Rasmussen,  1986;  Vicente  and  Rasmussen,  1992;  Lind, 
1993,  cited  in  Roth  and  Mumaw,  1995).  It  depicts  goals  and  functions  hierarchically.  For  each  function,  it  defines  higher- 
level  goals  and  lower-level  functions  (i.e.,  paths  for  accomplishing  the  goals).  This  analysis  provides  the  underpinning  for 
defining  personnel  tasks.  Each  function  identified  represents  a  decision  point  in  the  operator’s  role.  For  each  ftmction,  a 
set  of  questions  are  posed,  derived  from  an  operator’s  decision-making  model  (Rasmussen,  1986).  They  investigate  the 
information  requirements  and  control  capabilities  for  supporting  the  cognitive  activities  involved  in  ensuring  the  operation 
of  that  function.  The  questions  address  the  following  topics:  monitoring,  situation  awareness,  planning,  and  control.  The 
answers  define  the  HSI  requirements  that  operators  need  to  support  these  activities,  as  well  as  the  rationale  for  the  plant’s 
parameter  measurement  and  control  requirements  (provided  to  systems  engineers),  and  the  basis  of  design  descriptions  for 
displays. 

The  heavy  reliance  of  this  approach  upon  functional  analyses  has  advantages  and  disadvantages.  A  disadvantage  is  that 
such  methods  require  more  analysis  than  empirical  CTA  approaches.  An  advantage  is  that  the  methods  can  be  employed 
for  new  systems  with  no  close  current  analogies.  Thus,  function-based  CTA  methods  can  be  used  when  empirical  methods 
cannot.  A  second  advantage  is  that  function-based  CTA  uses  very  structured,  pragmatic  methods.  It  does  not  require 
expertise  in  behavioral  science  and  can  be  undertaken  by  analysts  with  engineering  backgrounds;  this  can  be  a  benefit  for 
engineering  organizations  developing  advanced  human-machine  systems.  Hence,  Roth  and  Mumaw  (1995)  recommend 
function-based  CTA  for  first-of-a-kind  design  projects. 
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Cognitive  Simulation  CTA  -  Cognitive  simulation  constructs  a  computer-based  model  that  simulates  the  cognitive  activities 
required  to  perform  a  task.  The  computer  model  explicitly  describes  knowledge  requirements  and  decision  processes 
(Zachary,  Ryder,  Ross,  and  Weiland,  1992,  cited  in  Roth  and  Mumaw,  1995;  Hollnagel,  1992;  Roth,  Woods,  and  Pople, 
1992).  System  design  factors  can  be  entered  into  the  model,  such  as  the  type  of  information  presented,  its  rate  of 
presentation,  and  its  reliability,  so  that  when  the  simulation  is  run,  the  effects  on  cognitive  activities  (e.g.,  the  amount  of 
knowledge  and  processing)  can  be  determined.  Comparisons  can  be  made  for  alternative  designs,  and  changes  in  design 
can  be  entered  and  the  operator’s  and  system’s  response  observed.  In  addition,  simulations  can  be  valuable  research  tools 
for  studying  human  cognition.  Insights  can  be  obtained  by  revealing  differences  between  human  performtmce  and 
performance  predicted  by  the  computer  model.  Cognitive  simulation  techniques  require  the  analyst  to  have  an 
understanding  of  cognition,  so  that  relevant  dimensions  of  human  performance  can  be  modeled.  Representing  information 
processing  in  a  computer-based  model  requires  a  background  in  software  development. 

The  following  describes  the  System  Response  Generator  (SRG)  (Hollnagel,  1992),  a  computer-based  simulation  tool 
developed  to  support  system  design.  It  illustrates  some  important  components  of  computer-based  simulations  and  their 
functions.  SRG  systematically  examines  the  possible  ways  in  which  a  scenario  can  develop  as  a  result  of  an  operator’s 
interactions  with  the  system.  It  is  intended  to  “...clarify  precisely  how  the  configuration  of  the  physical  system  and  the 
[user]  interface  may  sometimes  transform  human  erroneous  actions  into  accidents”  (Hollnagel,  1992,  p.  216).  SRG  has  four 
modules:  (1)  the  event  driver,  which  triggers  events  and  controls  the  progress  of  the  simulation  from  break  point  to  break 
point,  (2)  the  process  response  generator,  which  simulates  the  response  of  the  controlled  system  (e.g.,  an  aircraft)  as  it  reacts 
to  events  and  control  inputs  from  the  operator,  (3)  the  operator  response  generator,  which  simulates  the  perceptual  and 
decision-making  processes  of  the  operator  in  response  to  the  outputs  of  the  process  response  generator,  and  (4)  the  response 
interpreter,  which  examines  the  output  from  the  operator  response  generator  and  determines  the  implications  of  the 
operator’s  actions  for  the  event  driver.  Thus,  the  cognitive  simulation  (i.e.,  the  operator  response  generator)  is  immersed  in 
a  larger  simulation  that  includes  the  system  being  controlled  and  the  environment. 

Cognitive  simulations  offer  several  benefits  over  empirical  approaches  (Hollnagel,  1992): 

•  Events  or  operator  actions  that  do  not  lead  to  anticipated  consequences  may  go  unnoticed  unless  special  efforts  are 
undertaken  to  observe  them.  (Empirical  techniques  often  rely  on  well-defined  procedures  for  capturing  data. 

When  a  system’s  or  human’s  behavior  goes  outside  the  analyst’s  assumptions,  capturing  data  can  become  difficult.) 

•  Extremely  long  observations  may  be  needed  to  obtain  reliable  data  on  low-frequency  events.  (Although  scenarios 
can  be  created  that  increase  the  likelihood  of  specific  behaviors,  they  require  time  to  set  up  and  execute. 
Computer-based  scenarios  can  be  created  and  run  faster  than  scenarios  involving  real  people.) 

•  The  scope  for  generalizing  results  is  often  limited.  (Computer-based  simulations  test  a  broader  range  of 
conditions,  thus  increasing  the  generality  of  the  results.) 

•  Incompatible  classification  schemes  for  tasks,  actions,  causal  factors,  and  contributing  factors  may  interfere  with 
the  transfer  or  translation  of  data  to  new  applications.  (Due  to  the  complexity  of  dynamic  human-machine 
systems,  it  may  be  difficult  to  apply  classification  schemes  consistently  within  or  across  scenarios.  Computer-based 
simulation  gives  greater  control  over  causal,  contributing,  tmd  response  factors,  allowing  more  consistent  use  of 
classification  schemes.) 

•  The  exact  boundary  between  acceptable  performance  and  failure  may  be  difficult  to  define  for  non-proceduralized 
tasks.  (Computer-based  simulation  can  provide  greater  control  over  scenario  variables,  facilitating  the 
specification  of  criteria  for  acceptable  performance.) 
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Roth  et  al.  (1992)  describe  two  potential  disadvantages  of  cognitive  simulations: 

•  A  cognitive  simulation  may  be  a  continually  evolving  tool  due  to  such  factors  as  the  breadth  of  cognitive  activities 
in  a  particular  domain,  the  expanding  knowledge  base  in  cognitive  science,  and  pragmatic  factors  in  large  software 
development  projects.  Hence,  it  may  be  difficult  to  see  a  cognitive  simulation  as  a  finished  system. 

•  Cognitive  simulations  provide  examples  of  concepts  about  human  cognition,  but  are  not  the  concepts  themselves. 
They  should  be  seen  as  tools  for  modeling  cognitive  performance  rather  than  the  actual  model.  Much  value  can  be 
obtained  by  comparing  predictions  with  actual  performance. 

Finally,  the  initial  expense  of  developing  a  cognitive  simulation  may  be  high,  so  this  approach  may  be  more  applicable  for 
large  design  projects.  However,  once  developed,  these  simulation  tools  can  be  used  to  evaluate  very  small  to  very  large 
design  changes. 

5.2.3  Staffing  Analysis 

Staffing  requirements  should  be  reviewed,  in  accordance  with  Element  5  in  Section  6  of  NUREG-071 1,  for  upgrades  that 
are  potentially  significant  to  plant  safety,  affect  the  role  of  the  personnel  or  their  tasks,  and  are  likely  to  change  staffing 
requirements.  The  scope  of  the  staffing  analysis  should  cover  demands  resulting  from  the  upgrade  and  its  interactions  with 
the  plant. 

Minimum  staffing  levels  are  initially  determined  during  the  plant’s  design,  often  in  conjunction  with  the  CR  design. 
However,  a  licensee  may  establish  policies  for  higher  staffing  levels  based  on  special  operational  activities  (e.g.,  operational 
maneuvers;  on-line  testing)  or  other  conditions  that  may  impose  unusual  burdens  on  CR  personnel  (e.g.,  lack  of  some  HSI 
components).  Most  plant  and  HSI  upgrades  are  intended  to  reduce  demands  on  operators.  However,  changes  in  the  plant 
or  the  HSI  can  increase  demands  on  personnel,  especially  during  transients.  If  these  demands  are  sufficiently  high,  the 
adequacy  of  CR  staffing  should  be  considered.  Upgrades  could  affect  the  number  of  personnel,  and  also  their  required 
expertise. 

Number  of  Personnel 


The  number  of  personnel  needed  in  the  CR  may  be  affected  by  factors  that  increase  workload  or  compromise  the  crew’s 
performance.  For  example,  the  operators’  roles  may  be  changed  if  they  must  perform  new  or  changed  functions.  Then,  the 
assumptions  underlying  the  original  staffing  levels  may  no  longer  be  valid,  and  more  people  may  be  needed. 

Even  if  the  operator’s  functions  remain  unchanged,  changes  in  their  tasks  may  alter  the  required  staffing  levels.  Additional 
personnel  may  be  needed  if  the  frequency  or  difficulty  of  the  tasks  increase,  such  that  they  cannot  be  performed  effectively 
by  the  current  staff.  This  may  occur  when  an  HSI  upgrade  increases  the  amount  of  information  to  be  monitored,  the 
number  of  controls  to  be  operated,  the  frequency  of  needed  monitoring  or  control  actions,  or  decreases  the  accessibility  of 
information  and  controls.  It  may  also  occur  when  mental  workload  increases,  such  as  when  a  modification  to  a  system 
increases  the  plant’s  complexity  and  increases  demands  for  situation  awareness  and  response  planning. 

Staffing  requirements  may  also  be  affected  by  upgrades  affecting  the  coordination  of  activities  between  personnel;  examples 
include  those  that 
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•  Decrease  the  availability  of  operators,  as  when  the  upgrade  increases  the  demands  for  operators  to  perform  tasks 
outside  the  CR  (e.g.,  fire  brigade)  or  away  fi-om  its  main  control  area  (e.g.,  tasks  at  the  back  panels). 

•  Increase  the  demands  for  operators  to  coordinate  work  activities  (e.g.,  changes  that  increase  the  need  for  personnel 
to  gather  and  coordinate  information  and  control  actions). 

•  Decrease  the  ability  of  personnel  to  coordinate  work  activities  in  the  CR  (e.g.,  changes  such  as  background  noise 
that  interfere  with  speech,  or  interfere  with  people’s  line-of-sight,  such  as  layout  changes  or  installing  ec^uipment 
that  obscures  vision). 

Backgrounds  of  Personnel 

The  required  backgrounds  of  staff  may  be  affected  by  any  change  that  requires  them  to  have  different  knowledge,  skills,  or 
abilities  than  those  personnel  currently  possess.  For  example,  an  upgrade  may  require  balance-of-plant  operators  to  have  a 
more  advanced  understanding  of  thermodynamics  than  they  currently  have.  Also,  the  design  of  the  HSI  may  require  CR 
operators  to  have  special  skills  in  human-computer  interactions. 

NUREG-071 1  provides  criteria  for  analyzing  staffing.  It  states  that  the  analysis  should  be  iterative.  Initial  staffing  goals 
should  be  reviewed  and  modified  as  analyses  associated  with  other  NUREG-071 1  elements  are  completed. 

5.2.4  Human  Reliability  and  Design  Basis  Considerations 

The  purpose  of  human  reliability  analysis  (HRA)  is  to  evaluate  the  potential  for,  and  mechanisms  of,  human  error  that  may 
affect  plant  safety.  It  is  an  essential  element  in  achieving  the  HFE  design  goal  of  providing  operator  interfaces  that  will 
minimize  operators’  errors,  and  will  allow  their  detection  and  mitigation.  HRAs  focus  on  risk-important  human  actions, 
those  actions  which  may  challenge  plant  safety  if  performed  incorrectly.  The  HRA  review  should  be  conducted  in 
accordance  with  Element  6  in  Section  7  of  NUREG-071 1,  for  all  upgrades  that  are  safety  significant,  affect  the  role  of 
personnel,  and  may  affect  tasks  previously  identified  as  risk  important,  cause  existing  tasks  to  become  risk  important,  or 
create  new  risk-important  tasks.  The  scope  of  the  human  reliability  analysis  should  address  personnel  actions  resulting 
from  the  upgrade  and  its  interactions  with  the  rest  of  the  plant. 

HRAs  are  based  on  many  considerations,  such  as  plant  failure  modes  and  the  role  of  personnel  in  recovering  from  failures. 
For  example,  as  part  of  the  defense-in-depth  design  philosophy  of  NPP  design,  operators  are  credited  with  performing 
certain  acts,  such  as  initiating  the  operation  of  plant-protection  systems  when  they  fail  to  operate  automatically.  HRAs 
should  be  based  on  an  understanding  of  the  causes  and  modes  of  human  error,  such  as  those  described  in  this  report;  careful 
analysis  is  especially  important  for  actions  determined  to  be  risk  important  in  the  HRA. 

When  upgrading  systems  or  the  HSI,  the  potential  effects  of  these  modifications  on  the  assumptions  underlying  the  existing 
HRA  should  be  considered.  For  example,  HRAs  often  assume  that  operators  can  detect  a  fault  and  initiate  a  manual 
response  within  a  specific  time.  Changes  to  the  plant’s  systems  may  change  the  bases  of  these  assumptions.  Some  may 
simplify  the  roles  of  personnel,  making  the  plant  safer;  others  may  complicate  the  operator’s  role.  Thus,  as  plant  systems 
become  more  complex,  as  through  increased  automation,  the  operator’s  role  in  detecting  malfunctions  may  become  more 
challenging;  that  is,  it  may  be  more  difficult  to  determine  that  the  system  is  operating  incorrectly.  Also,  the  actions 
required  to  respond  to  these  conditions  may  be  more  complex,  introducing  new  opportunities  for  error.  Stubler,  Higgins, 
and  Kramer  (2000)  describe  events  that  occurred  in  NPPs  due  to  errors  by  operators  and  maintenance  personnel  while 
performing  at-power  maintenance  on  digital  systems. 
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Changes  in  the  HSI  also  may  affect  the  HRA  assumptions  about  personnel  performance.  For  example,  an  HRA  may  take 
credit  for  a  particular  manual  action.  If  the  relevant  controls  and  displays  are  upgraded  with  computer-based  HSI 
technologies,  these  tasks  may  be  altered.  The  computer-based  HSI  may  lead  to  errors  in  navigating  a  display  system  to  find 
the  correct  controls  and  displays.  The  execution  of  control  actions  using  soft  controls  also  may  be  prone  to  special  types  of 
errors  (Stubler,  O’Hara,  and  Kramer,  2000).  These  factors  decrease  the  probability  that  the  required  action  will  be 
completed  correctly  within  the  required  time  limit. 

Risk-important  actions  should  be  explored  when  examining  the  effects  of  a  plant  system  or  HSI  upgrade  on  the  HRA.  The 
following  questions  should  be  considered; 

•  Are  the  HRA  assumptions  valid  for  the  upgraded  design? 

•  Will  changes  in  plant  systems  or  the  HSI  increase  the  likelihood  of  mistakes  (e.g.,  errors  from  inappropriate  plans 
based  on  incorrectly  assessing  the  plant’s  state)? 

•  Will  changes  in  the  HSI  increase  the  likelihood  of  slips  (e.g.,  errors  in  executing  actions)? 

•  Will  the  consequences  of  these  errors  exceed  the  bounds  established  in  the  HRA? 

5.2.5  Summary  of  Analysis  Phase 

The  analysis  phase  addresses  human  performance  considerations  associated  with  identifying  fimctions,  tasks,  staffing,  crew 
coordination  considerations,  and  human  reliability  factors.  The  analysis  of  these  topics  should  be  commensurate  with  the 
scope  of  the  upgrade  and  its  potential  consequences  to  personnel  p)erformance  and  plant  safety.  Functional  requirements 
should  be  examined  to  identify  and  evaluate  changes  in  plant  safety  functions  that  may  occur  from  the  upgrade.  They 
should  identify  functions  that  are  new  or  changed.  Also,  the  allocation  of  functions  between  personnel  and  automation 
should  be  evaluated.  These  changes  in  functions  and  allocations  partly  establish  a  basis  for  new  personnel  tasks. 

Task  analysis  looks  at  the  specific  characteristics  of  personnel  tasks  to  establish  a  basis  for  design  requirements  for 
upgrades.  Highly  proficient  users  often  employ  special  strategies  to  cope  with  work  demands.  This  section  described  three 
categories  of  strategies;  task  automaticity,  information  chunking,  and  HSI  tailoring,  all  of  which  should  be  identified  and 
analyzed  in  the  current  work  environment.  The  new  design  should  either  have  features  to  support  these  strategies,  or 
should  present  control  and  display  capabilities  in  ways  that  eliminate  the  need  for  them.  Traditional  techniques  of  task 
analysis  emphasize  observable  behavior.  In  some  cases,  a  more  detailed  analysis  of  problem  solving  and  decision  making 
requirements  may  be  needed.  This  section  described  three  categories  of  cognitive  task  analyses  (empirical,  fimction-based, 
and  cognitive  simulation)  that  support  the  analysis  of  the  cognitive  activities.  The  failure  of  a  licensee  to  address  cognitive 
tasks  and  strategies  during  task  analysis  may  indicate  that  proper  attention  was  not  given  to  HFE  considerations  during  the 
requirements-analysis  process  for  upgrades. 

Finally,  a  review  of  the  requirements-analysis  phase  of  an  upgrade  should  consider  staffing  and  human  reliability. 

Upgrades  generally  are  intended  to  reduce  personnel’s  workload  and  enhance  reliability  and  safety.  However,  upgrades 
sometimes  may  be  at  variance  with  the  assumptions  underlying  earlier  analyses.  When  upgrades  affect  risk-important 
activities  or  impose  new  demands  on  the  number  and  qualifications  of  plant  personnel,  then  it  may  be  appropriate  to  review 
them  in  detail. 
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8.3  Interface  Design  Phase 

The  interface  design  phase  described  in  NUREG-071 1  addresses  the  design,  development,  and  testing  of  the  various 
interfaces  between  personnel  and  the  plant.  This  section  addresses  the  HFE  considerations  for  the  design  of  the  following 
aspects  of  upgrades: 

•  HSI 

•  Procedures 

•  Training. 

5.3.1  HSI  Design 

The  purpose  of  this  review  is  to  ensure  that  design  requirements  were  appropriately  translated  into  the  detailed  HSI  design 
by  systematically  applying  HFE  principles  and  criteria.  This  review  may  evaluate  both  the  HSI  design  process,  and  the 
product  of  that  process.  The  process  and  criteria  for  conducting  a  thorough  HFE  review  of  a  new  HSI  design  is  set  out  in 
Element  7,  Human-System  Interface  Design,  in  Section  8,  NUREG-071 1.  This  review  should  be  conducted  for  all  upgrades 
that  are  significant  to  safety,  affect  the  role  or  tasks  of  personnel,  and 

•  Change  existing  HSI  components  used  for  risk-important  tasks,  or 

•  Provide  new  HSI  components  for  risk-important  tasks. 

The  scope  of  the  HSI  design  review,  specified  in  Section  8.4. 1.1  of  NUREG-071 1,  may  be  limited  to  the  upgrade  and  its 
interactions  with  the  other  components  of  the  HSI.  Inputs  to  the  HSI  design  process  specified  in  Section  8.4. 1.2,  may  be 
limited  to  those  from  HSI  design  process  analyses  that  are  applicable  to  the  upgrade.  Similarly,  the  HSI  design  procedures, 
specified  in  Section  8.4. 1.3,  may  be  limited  to  those  design  process  analyses  applicable  to  the  upgrade. 

The  following  is  a  discussion  of  HSI  design  considerations  that  are  particular  to  HSI  upgrades.  They  were  identified  by 
reviewing  literature  pertaining  to  complex  human-machine  systems,  and  from  reviews  of  industry  experiences.  They  should 
be  dealt  with,  along  with  the  other  considerations  in  Section  8.4  of  NUREG-071 1,  during  HSI  design  reviews  for  upgrades. 
The  following  are  discussed:  automaticity;  consistency  between  new  and  replaced  HSI  components;  consistency  between 
new  HSI  components  and  the  rest  of  the  HSI;  the  functional  integration  of  HSI  components;  and  crew  coordination  and 
cooperation. 

Design  for  Automaticity 

Tasks  that  may  benefit  from  automaticity,  such  as  those  that  must  be  performed  quickly  and  do  not  have  high  negative 
consequences,  should  be  addressed  by  developing  user  interfaces  that  impose  minimal  demands  upon  attention  or  mental 
processing  to  operate  them.  They  should  be  designed  according  to  the  two  high-level  HSI  design  principles  for  secondary 
task  control  that  suggest  ways  to  minimize  cognitive  and  response  workload  (NUREG-0700,  Appendix  A;  also  Appendix  A 
of  this  report).  For  example,  they  should  not  require  a  great  degree  of  fine  motor  skill,  such  as  positioning  a  cursor  on  a 
small  target,  or  of  physical  effort,  such  as  moving  a  switch  with  a  high  activation  force,  nor  should  they  impose  high 
demands  on  working  memory,  such  as  requiring  the  user  to  remember  numerical  values  or  identification  codes.  Similarly, 
they  should  not  impose  high  demands  on  long-term  memory,  such  as  requiring  the  user  to  remember  an  unusual  pattern  of 
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actions.  The  means  of  operating  these  user  interfaces  should  be  consistent  with  population  stereotypes.  In  addition,  user 
interfaces  should  be  designed  consistently  so  that  actions  with  similar  goals  can  be  performed  the  same  way  using  interfaces 
that  appear  similar. 

To  encourage  consistency  in  user  interfaces,  the  design  organization  should  develop  its  own  design  standards.  Their 
development  should  be  based  on  HFE  reviews  of  HSI  technologies,  design  approaches,  and  user’s  requirements.  The 
standards  should  address  the  appearance,  function,  and  means  of  operation  of  user  interfaces,  and  should  be  documented  for 
the  design  organization,  as  described  in  Section  8.4.2. 1,  HFE  Design  Guidance  Development,  NUREG-071 1. 

The  effectiveness  of  design  approaches  for  supporting  automaticity  should  be  evaluated  through  performance-based  tests 
during  the  design  process.  They  should  ensure  that  the  user  interfaces  can  be  operated  quickly,  accurately  and  without 
apparent  difficulty. 

Separate  design  techniques  should  be  used  for  those  tasks  for  which  automation  should  be  discouraged,  because  it  may 
result  in  errors  with  significant  negative  consequences.  A  review  of  soft  controls  used  in  complex  human-machine  systems 
found  that  these  interfaces  are  especially  prone  to  the  following  categories  of  slips:  unintended  actuation,  description  errors, 
capture  errors,  mode  errors,  misordered  components  of  an  action  sequence,  and  loss-of-activation  errors.  Descriptions  of 
these  categories  and  specific  design  approaches  for  protecting  against  errors  are  given  in  Stubler,  O’Hara,  and  Kramer  . 
(2000).  Performance-based  tests  should  be  conducted  to  identify  the  types  of  errors  that  may  be  made,  to  evaluate  the 
effectiveness  of  alternative  means  of  protection  against  them,  and  to  demonstrate  the  effectiveness  of  the  final  design. 

Design  for  Consistency  Between  New  and  Replaced  HSI  Components 

When  the  HSI  is  upgraded,  such  as  when  a  component  is  replaced,  some  of  the  design  characteristics  that  support  user’s 
performance  may  change,  sometimes  in  ways  that  have  safety  significance.  For  example,  an  indicator  light  on  a  controller 
could  represent  normal  status  in  the  original  design,  but  represent  a  serious  condition  in  the  replacement.  If  an  operator 
applied  knowledge  of  the  old  controller  when  viewing  the  new  one,  an  important  problem  may  not  be  diagnosed.  Also,  an 
action  that  produces  a  small  adjustment  in  feedwater  flow  in  the  old  controller  may  produce  a  large  one  in  the  new 
controller;  applying  the  old  skill  to  the  new  controller  may  upset  feedwater  flow.  In  such  cases,  the  old  and  new  HSI 
components  may  be  considered  to  have  poor  consistency.  An  HFE  review  should  attempt  to  identify  such  inconsistencies, 
their  potential  consequences,  and  the  need  for  remediation. 

A  model  of  transfer  (e.g.,  Osgood,  1949)  is  often  used  as  a  framework  for  describing  these  effects  (Holding,  1987;  Swezey 
and  Llaneras,  1997;  Sawyer,  Pain,  Van  Cott,  and  Banks,  1982).  Positive  and  negative  transfer,  respectively,  refer  to  the 
facilitative  and  inhibitory  effects  of  prior  learning  upon  performance  in  new  (transfer)  circumstances.  A  positive  transfer  of 
training  is  said  to  have  occurred  if  the  experience  of  learning  to  use  the  old  component  facilitates  learning  the  new 
component.  For  example,  if  a  new  feedwater  controller  is  installed  in  the  CR  and  the  operators  who  were  skilled  in  using 
the  old  controller  learn  to  use  the  new  controller  faster  than  operators  who  did  not  use  the  old  one,  then  a  positive  transfer 
of  training  has  occurred. 

In  some  cases,  prior  learning  with  an  old  component  may  inhibit  the  ability  to  learn  the  new  one.  This  negative  transfer  of 
training  can  increase  training  times,  decrease  response  time  when  using  the  upgraded  HSI,  and  lead  to  errors. 

Transfer  conditions  are  usually  described  as  a  function  of  the  similarity  of  stimulus  and  response.  The  Osgood  model 
represents  gradients  of  similarity  between  the  stimuli  and  responses  of  the  training  and  operational  (transfer)  environments 
using  a  three-dimensional  surface.  Stimulus  similarity  is  presented  on  one  axis,  response  similarity  on  a  second  axis,  and 
the  amount  and  direction  of  transfer  (i.e.,  positive  versus  negative)  on  the  third  dimension.  The  effect  of  variations  in  the 
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required  responses  changes  from  maximum  positive  transfer  when  the  responses  are  nearly  identical,  to  zero  (no  transfer), 
to  negative  transfer  when  antagonistic  (i.e.,  opposite)  responses  are  required.  When  responses  are  identical  and  the  stimuli 
are  varied,  transfer  drops  to  zero  as  the  stimuli’s  similarity  decreases.  However,  with  antagonistic  responses,  transfer 
increases  from  negative  to  zero  as  the  stimuli’s  similarity  decreases. 

Attempts  to  replicate  the  predictions  of  the  Osgood  model  in  experiments  have  had  mixed  results.  While  the  model 
characterizes  transfer  performance  with  simple  tasks,  such  as  learning  verbal  paired  associates,  the  transfer  surface  is 
inadequate  for  complex  tasks  (Swezey  and  Llaneras,  1997).  Contradictory  results  were  obtained  in  predicting  negative 
transfer  (Holding,  1987).  Numerous  studies  of  real-world  learning  also  contradicted  the  Osgood  model’s  prediction  of 
positive  transfer  by  demonstrating  that  transfer  may  increase  (rather  than  decrease)  by  reducing  the  fidelity  of  a  training 
simulator,  thus  reducing  the  similarity  between  the  training  and  the  transfer  environment  (Swezey  and  Llaneras,  1997). 

One  problem  with  applying  this  model  to  the  complex  tasks  performed  in  NPP  CRs  is  the  difficulty  in  defining  the  degree  of 
similarity  of  stimuli  or  responses  which  may  have  many  important  dimensions  for  personnel  performance.  For  example,  a 
simple  HSI  component,  such  as  display,  may  be  described  by  its  position,  shape,  size,  color,  information  content, 
informational  change,  and  spatial  relationships.  The  response,  such  as  operating  a  switch,  may  be  described  in  terms  of 
discrete  versus  sequential  actions  involving  identification,  discrimination,  reading,  and  movement  (Sawyer  et  al.,  1982). 
Further,  the  work  environment  may  contribute  other  dimensions  to  the  stimuli.  For  example,  a  simple  action  such  as 
operating  a  switch  may  be  associated  with  stimuli  such  as  procedure  steps,  verbal  instructions,  alarms,  and  various 
indications  on  a  control  panel.  The  response,  flipping  the  switch,  may  also  be  associated  with  sequential  relationships  with 
other  controls  and  displays,  and  the  operator’s  intentions  and  expectations.  Because  stimuli  and  responses  may  vary  in 
many  dimensions,  it  is  difficult  to  define  the  degree  of  similarity  of  different  conditions,  imposing  practical  obstacles  to 
applying  the  transfer  model. 

Another  problem  is  that  much  of  the  empirical  work  and  resulting  HFE  guidance  pertains  to  transfer  effects  between  a 
training  environment  and  an  operational  environment,  primarily  addressing  ways  to  modify  the  user  interface  to  enhance 
positive  transfer.  However,  for  reviewing  HSI  upgrades,  the  primary  concern  is  to  identify  and  assess  factors  that  contribute 
to  negative  transfer,  for  which  there  is  less  guidance. 

Sawyer  et  al.  (1982)  reviewed  the  transfer  literature  and  developed  a  set  of  initial  principles  on  NPP  CR  modifications. 
Although  not  comprehensive,  these  principles  are  consistent  with  current  transfer  theory,  and  provide  a  basis  for 
considering  inconsistencies  between  the  original  and  upgraded  HSI  that  may  affect  personnel  performance.  Thus,  these 
considerations  deserve  attention  when  reviewing  an  HSI  upgrade;  the  following  are  derived  by  Sawyer  et  al.  (1982): 

(1)  Negative  transfer  may  be  produced  when  responses,  which  conflict  with  the  original  ones,  are  introduced  into  the 
modified  HSI  design. 

(2)  A  possible  exception  to  consideration  1,  is  when  the  original  controls  and  displays  conflict  with  population 
stereotypes,  or  are  not  designed  consistently  with  each  other.  Then,  personnel  performance  may  already  be  affected 
by  these  inconsistencies,  and  modifying  the  responses  may  improve  it. 

(3)  Positive  transfer  usually  occurs  when  responses  are  unchanged  between  the  original  and  modified  components, 
even  if  the  stimuli  change.  For  example,  if  the  new  HSI  component  looks  different  but  responds  in  the  same  way, 
personnel  often  adapt  quickly  (i.e.,  positive  transfer  occurs). 

(4)  The  greatest  amount  of  positive  transfer  is  generally  produced  when  the  old  and  new  component  have  conceptual 
similarity.  To  the  degree  that  their  stimuli  and  responses  are  similar,  positive  transfer  will  usually  occur. 
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(5)  If  both  the  stimuli  and  responses  are  different  between  the  old  and  new  conditions,  little  or  no  transfer  can  be 
expected. 

(6)  There  is  little  evidence  to  either  support  or  refute  the  notion  that  increased  stress  causes  prior  patterns  of  response 
to  regress. 

Transfer  from  the  old  HSI  components  to  new  ones  may  be  mediated  by  several  factors.  The  following  are  based  on  review 
considerations  offered  by  Sawyer  et  al.  (1982): 

(1)  The  amount  of  positive  or  negative  transfer  partly  depends  upon  the  degree  of  similarity  or  difference  between  old 
and  new  stimuli  or  responses.  However,  all  the  factors  that  contribute  to  these  similarities  and  differences  may  not 
be  known. 

(2)  Positive  transfer  may  be  enhanced  by  enforcing  the  user’s  understanding  of  how  stimuli  and  responses  relate  to  the 
function  of  the  HSI  component  and  the  associated  plant  system.  [This  consideration  is  based  on  the  concept  of 
mediation  (Sawyer  et  al.,  1982).] 

(3)  Positive  transfer  may  be  enhanced  when  verbal  cues  or  names  associated  with  the  HSI  changes  can  be  learned  by 
users.  [This  consideration  is  based  on  the  concept  of  predifferentiation  (Sawyer  et  al.,  1982).] 

Schlager  et  al.  (1989)  have  a  different  perspective  on  transferring  knowledge  and  skills.  They  indicate  that  in  complex  task 
environments,  such  as  control  room  settings,  personnel  employ  several  cognitive  skills,  such  as  chunking  of  information 
and  automaticity  of  actions  (see  Sections  4.3.1  and  4.3.2),  to  enable  them  to  overcome  information  processing  limitations. 
Many  of  these  skills  are  supported  by  particular  characteristics  of  the  HSI  design,  such  as  its  spatial  arrangement  and 
coding  schemes.  When  the  HSI  is  upgraded,  transfer  performance  is  enhanced  to  the  extent  that  the  new  configuration 
supports  the  existing  strategies. 

This  can  be  illustrated  with  four  cases.  In  the  first  case,  the  design  characteristics  allow  these  strategies  to  be  applied  in  the 
same  way  in  the  original  and  upgraded  HSI  environments;  hence,  personnel  readily  adapt  to  the  change  and  transfer  is 
high.  In  the  second,  the  strategies  used  in  the  original  configuration  are  applicable  but  must  be  slightly  modified  to 
accommodate  the  upgrade  so  that  transfer  declines  a  little.  For  example,  personnel  may  select  and  sort  information  the 
same  way  but  use  different  HSI  characteristics  when  doing  so  (i.e.,  an  air  traffic  controller  sorts  inbound  and  outbound 
aircraft  by  the  color  of  the  icons  rather  than  by  their  shape).  In  the  third  case,  the  design  characteristics  do  not  support  the 
user’s  strategies  and  new  skills  must  be  developed.  This  may  be  very  disruptive  to  personnel  performance,  especially  if 
users  must  discover  new  strategies  on  their  own.  If  they  do  not,  performance  may  decline. 

In  the  fourth  case,  the  HSI  undertakes  some  of  the  operators’  task  of  information  processing.  For  example,  a  new  air  traffic 
control  display  system  may  automatically  separate  inbound  and  outbound  aircraft  so  that  users  are  not  required  to  do  this 
mentally.  The  upgrade  organizes  information  in  a  way  that  is  consistent  with  the  user’s  task,  while  reducing  some  of  the 
user’s  cognitive-processing  demands.  In  addition  to  reducing  the  mental  workload  of  experienced  users,  this  approach 
benefits  newer  users  by  allowing  them  to  achieve  high  levels  of  performance  without  having  to  learn  the  special 
information-grouping  skills  of  the  experts.  An  example  from  air  traffic  control  is  given  by  Schlager  et  al.  (1989); 

The  Inset  Display  is  capable  of  presenting  graphically  a  conceptual  chunk  of  aircraft  [traffic]  such  as  ’all  aircraft 
landing  in  a  particular  airport’  while  filtering  out  all  others.  Thus,  the  potential  exists  for  these  features  to  facilitate 
the  performance  of  full  performance  level  controllers  as  well  as  new  [personnel]  by  greatly  reducing  the  amount  of 
information  they  must  keep  in  working  memory,  (p.  48) 
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These  benefits  can  only  be  achieved  after  users  learn  these  new  features  and  incorporate  them  into  their  work.  Schlager  et 
al.  note  that  careful  analysis  and  design  may  be  necessary  to  ensure  that  the  way  information  is  organized  in  the  displays  is 
consistent  with  the  ways  in  which  users  organize  the  information.  This  process  may  be  a  focus  area  of  an  HFE  review. 

Design  for  Consistency  Between  New  HSI  Components  and  the  Rest  of  the  HSI 

HSI  upgrades  often  are  installed  in  NPPs  on  a  component-by-component  basis  (O’Hara,  Stubler,  and  Higgins,  1996).  For 
example,  in  a  transition  to  digital  technologies,  the  old  analog  component  is  removed  from  the  control  panel  and  a  digital 
HSI  component  is  installed.  However,  many  replacement  and  upgrade  products  are  generically  designed,  such  as  flat  panel 
displays  and  input  devices,  and  each  manufacturer  may  have  different  standards  for  them.  Therefore,  after  several  upgrades 
have  been  installed,  the  HSI  may  contain  many  similar-looking  user  interfaces  with  different  conventions  for  presenting 
information  and  different  mechanisms  for  accessing  and  controlling  it.  This  can  affect  the  overall  consistency  of  the  HSI. 
For  example,  information  coding  conventions  may  have  different  means  on  different  devices,  and  the  correct  method  of 
operating  one  device  may  be  incorrect  for  a  similar  one. 

In  this  setting,  personnel  may  make  frequent  transitions  between  the  various  HSI  components.  Norman  (1983)  describes 
errors  that  occur  when  user  interfaces  lack  consistency.  When  users  lack  the  knowledge  to  properly  operate  some  aspect  of 
an  interface,  they  may  try  to  do  so  correctly  through  analogy  with  other,  similar  aspects  of  the  device.  This  derivation  may 
be  entirely  unconscious.  Errors  occur  when  the  user  derives  an  action  sequence  that  is  inappropriate  because  the  user 
interface  has  a  command  structure  that  differs  from  (is  inconsistent  with)  that  on  which  the  analogy  was  based,  even  though 
the  commands  appear  to  be  related  and  share  a  common  description  of  purpose,  action,  and  part  of  the  command  format. 
Norman  states  that  in  addition  to  affecting  input,  this  phenomenon  may  influence  the  user’s  behavior  when  reading 
displays. 

Tanaka  et  al.  (1991)  describe  this  as  a  problem  of  alternating  use  of  interfaces,  and  contrast  it  with  the  transfer  paradigm  in 
which  the  user  is  exposed  to  one  set  of  stimuli  (i.e.,  a  user  interface)  and  then  another,  but  does  not  return  to  the  first  set. 
This  situation  occurs  when  an  old  user  interface  is  replaced  with  a  new  one.  In  the  alternating  use  model,  the  user 
continually  switches  between  various  user  interfaces,  as  when  an  HSI  contains  a  variety  of  user  interfaces.  The  transfer 
model  predicts  that  performance  will  be  enhanced  when  transferring  between  two  interfaces  in  which  the  responses 
(methods  of  operating  the  interface)  are  similar.  For  example,  if  stimulus  1  (user  interface  1)  and  stimulus  2  (user  interface 
2)  differ  but  their  required  responses  are  similar,  then  performance  will  be  enhanced  when  transferring  from  1  to  2. 
However,  Tanaka  et  al.  predicted  that  when  the  user  alternates  between  user  interface  1  and  user  interface  2,  this  similarity 
in  response  will  impair  rather  than  enhance  performance;  this  high  overlap  of  methods  used  in  the  interfaces  will  cause 
errors  and  slow  execution  times.  Performance  suffers  because  the  user  cannot  remember  which  methods  correspond  to 
which  interface. 

Eberts  (1994)  provides  the  following  example.  The  DOS  computer  operating  system  is  command  based;  a  directory  is 
displayed  by  entering  the  command  “DIR.”  The  VMS  operating  system  is  also  command  based,  but  users  can  enter  as 
many  characters  as  they  wish,  as  long  as  the  minimum  needed  to  distinguish  the  command  from  others  is  provided.  For 
example,  to  display  a  directory  in  VMS  the  user  may  enter  the  minimum  character  string,  “DIR,”  the  maximum  character 
string,  “DIRECTORY,”  or  one  in  between  (e.g.,  “DIRECT’).  However,  in  DOS,  any  characters  beyond  “DIR”  are 
unacceptable  and  result  in  an  error  message.  Because  of  the  high  overlap  in  these  methods,  the  user  may  have  difficulty 
remembering  which  command  to  use  in  switching  between  the  two.  By  contrast,  the  transfer  paradigm  would  predict  that 
performance  would  be  enhanced. 

Kieras  (1988;  cited  by  Tanaka  et.  al.,  1991)  investigated  ways  in  which  a  natural-language  goals,  operations,  methods,  and 
selection  rules  language  (NGOMSL)  may  be  used  to  evaluate  aspects  of  human-computer  interaction  tasks.  Kieras’s  model 
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predicts  a  high  gain  in  user  performance  due  to  consistency  between  DOS  and  VMS.  However,  a  separate  model,  also  based 
on  NGOMSL  but  developed  by  Tanaka  et  al.,  predicts  a  decrement  due  to  what  they  consider  inconsistency.  As  another 
example,  Tanaka’s  model  predicts  that  a  user  should  encounter  very  little  difficulty  when  switching  between  the  DOS 
system  and  the  Apple  Macintosh  system  because  they  have  very  different  methods  for  performing  this  task  and  so  little 
confusion  should  occur.  Unlike  DOS,  the  Macintosh  operating  system  has  a  direct  manipulation  interface.  A  directory  is 
retrieved  by  using  a  mouse  to  double-click  on  a  folder  icon.  This  prediction  conflicts  with  those  based  on  the  Kieras  model 
and  the  transfer  paradigm. 

Tanaka  et  al.  used  a  NGOMSL  task  analysis  to  quantitatively  determine  consistency  between  interface  designs  that  were 
used  alternatively.  They  counted  the  number  of  changes  that  would  be  needed  to  change  the  methods  of  one  task  to  those  of 
another.  This  analysis  was  similar  to  that  used  by  Kieras  to  determine  gains  due  to  consistency.  However,  there  was  one 
important  difference;  Tanaka’s  analysis  treated  the  overlap  in  task  methods  in  a  different  way.  In  Tanaka’s  analysis,  if  two 
tasks  were  the  same  except  for  the  insertion  or  deletion  of  some  steps,  then  they  were  considered  inconsistent,  but  if  the  two 
task  methods  were  the  same  except  for  replacement  of  specific  words  in  the  steps,  then  they  were  considered  consistent. 
Tanaka  et  al.  tested  this  model  empirically.  Task  execution  times  were  faster  for  those  tasks  that  had  been  determined  to  be 
consistent  than  for  inconsistent  tasks.  In  addition,  these  differences  were  enhanced  when  users  returned  to  the  tasks  after  an 
absence. 

Tanaka  et  al.  conclude  that  the  practical  significance  of  their  model  and  experiments  was  that  consistency  can  be 
determined  only  within  the  context  of  the  tasks  used  and  their  sequencing.  The  same  set  of  interfaces  can  be  consistent  or 
inconsistent,  depending  upon  whether  use  is  transferred  from  one  interface  to  the  other,  or  is  alternated.  This  conclusion 
suggests  that  conventional  HFE  principles  of  consistency  and  standardization  of  the  interface  may  need  to  be  applied 
differently  in  a  computer-based  control  system.  In  particular,  HFE  reviews  should 

•  Distinguish  between  similarities  in  the  appearance  of  the  user  interfaces  and  the  methods  used  to  operate  them. 

•  Determine  which  user  interfaces  will  be  used  on  an  alternating  basis  by  the  same  person. 

Greater  attention  should  be  focused  on  interfaces  that  have  similar  methods  and  will  be  used  alternatively  by  the  same  user. 

Based  on  their  results,  Tanaka  et  al.  developed  the  text-editing  model  (TEM)  to  quantitatively  predict  the  effects  of  different 
kinds  of  computer  display  and  interaction-consistent  mappings  on  the  prolonged  performance  of  computer-based  tasks. 
Because  TEM  is  based  on  the  goals,  operations,  methods,  and  selection  rules  (GOMS)  model  (Card,  Moran,  and  Newell, 
1983),  it  can  be  readily  and  advantageously  applied  to  any  human-computer  interaction  situation  that  was  previously 
examined  by  GOMS.  This  model  is  being  refined  and  may  eventually  provide  a  useful  analytic  tool. 

Design  for  Functional  Integration  of  the  HSI 

A  problem  that  may  occur  when  plant  systems  and  HSI  components  are  upgraded  is  a  lack  of  apparent  functional 
integration.  This  describes  the  situation  in  which  different  control  or  display  devices  appear  to  be  functionally  related, 
when  in  fact  they  are  not.  It  may  occur  when  the  user  interfaces  of  the  HSI  do  not  provide  sufficient  information  about  the 
relationships  between  devices. 

The  following  example  is  from  a  chemical  plant.  Part  of  the  plant’s  alarm  system  was  replaced  with  a  digital  system. 
However,  some  signals  from  older  analog  systems  also  were  fed  into  it.  This  led  to  misunderstandings  by  the  operators, 
some  of  whom  incorrectly  assumed  that  any  alarm  appearing  on  the  digital  alarm  display  indicated  a  fault  with  one  of  the 
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digital  control  systems.  This  misunderstanding  led  to  difficulty  in  diagnosing  the  problem  when  the  alarms  were  generated 
by  the  analog  systems  (O’Hara,  Stubler,  and  Higgins,  1996). 

In  another  example,  a  computer-based  display  system  was  installed  as  a  CR  upgrade  to  help  operators  monitor  the  trip 
parameters  for  one  of  the  reactor’s  two  safety  shutdown  systems.  Operators  could  access  displays  that  indicated  the  current 
value  of  the  parameters  and  their  difference  from  the  trip  limits.  The  safety  shutdown  system  was  controlled  by  a  separate 
computer  and,  periodically,  the  trip  limits  had  to  be  manually  changed  to  reflect  changing  characteristics  of  the  reactor  core. 
However,  the  new  display  system  did  not  communicate  with  the  safety  shutdown  system’s  computer.  Consequently,  the  trip 
limits  had  to  be  manually  entered  into  both  computers  when  the  shutdown  system’s  trip  setpoints  were  changed.  It  was  not 
apparent  from  the  interface  that  these  computers  were  not  in  communication.  If  the  data  were  entered  incorrectly  into  the 
computer-based  display  system,  it  could  create  opportunities  for  mistakes;  operators  monitoring  the  reactor  trip  parameters 
might  be  unaware  that  the  margin  between  the  actual  value  and  the  trip  limit  did  not  represent  the  true  state  of  the  plant 
(Vicente  and  Bumes,  1995). 

When  a  plant  upgrade  is  developed,  the  analysis  of  design  requirements  should  identify  any  changes  in  the  degree  of 
integration  between  systems.  These  changes  should  be  clearly  indicated  in  the  HSI,  and  also  should  be  addressed  by 
training.  In  addition,  this  analysis  should  identify  cases  in  which  the  new  design  makes  systems  appear  to  be  functionally 
integrated  when  they  are  not.  The  HSI  design  should  ensure  that  the  true  relationship  is  accurately  conveyed  in  displays, 
procedures,  and  training. 

Design  for  Crew  Coordination  and  Cooperation 

The  design  of  an  HSI  upgrade  should  consider  the  effects  of  crew  coordination  discussed  in  Section  5.2.2,  Task  Analysis. 
General  review  guidance  for  workplace  design  is  given  in  NUREG-0700,  Rev.  1.  However,  computer-based  HSI 
technologies  can  affect  crew  coordination  in  complex  ways,  as  was  recognized  for  such  diverse  topics  as  operator-aiding 
systems  in  NPPs  (Dien  and  Montmayeul,  1995),  group-view  displays  for  NPP  CRs  (Stubler  and  O’Hara,  1996b),  computer- 
based  procedures  of  NPPs  (O’Hara,  Higgins,  Stubler,  and  Kramer,  2000),  and  design  of  ships’  bridges  (Hutchins,  1990). 

Dien  and  Montmayeul  state  that  the  HSI  design  should  be  compatible  with  the  organizational  structure  of  the  crew  and 
should  support  manageable  levels  of  workload  while  ensuring  plant  safety.  They  specifically  identify  the  following  topics  of 
concern:  using  HSI  design  to  support  the  crew’s  coordination,  and  the  flexible  allocation  of  tasks  between  members. 
Guidance  is  slowly  evolving;  meanwhile,  Dien  and  Montmayeul  recommend  using  expert  judgement  coupled  with  extensive 
testing  with  conceptual  prototypes. 

5.3.2  Procedure  Development 


Element  8  in  Section  9  of  NUREG-071 1  states  that  the  licensee’s  procedure  development  should  result  in  procedures  that 
support  and  guide  human  interactions  with  the  upgraded  HSI  and  plant  systems,  and  support  the  appropriate  response  to 
plant-related  events  and  activities.  Because  procedures  are  an  essential  component  of  HSI  design,  they  should  be  derived 
from  the  same  design  process  and  analyses  as  other  HSI  components  and  be  similarly  evaluated.  A  procedure  review  should 
ensure  that  HFE  principles  and  criteria  are  applied  along  with  all  other  design  requirements  to  develop  procedures  that  are 
technically  accurate,  comprehensive,  explicit,  easy  to  use,  and  validated.  This  review  includes  both  the  procedures  and 
procedure  development.  When  the  HSI  or  plant  systems  are  upgraded,  procedures  also  should  be  modified  to  reflect  these 
changes.  Procedures  should  be  developed  for  all  upgrades  that  are  significant  to  safety  and  affect  people’s  roles,  addressing 
all  personnel  tasks  affected  by  the  upgrade  and  its  interactions  with  the  rest  of  the  plant. 

The  following  describes  some  major  considerations  of  procedure  development  from  NUREG-071 1,  applicable  to  upgrades. 
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Content 


The  review  should  ensure  that  the  procedures  provide  acceptable  guidance  to  support  personnel  in  using  the  upgrades  to 
ensure  plant  safety.  Criterion  2  of  Section  9.4  of  NUREG-071 1  describes  the  bases  that  should  be  considered  in  developing 
procedures;  for  upgrades,  the  following  may  be  particularly  relevant: 

•  System-based  technical  requirements  and  specifications,  especially  where  plant  systems  were  modified  by  the 
upgrade 

•  Results  of  task  analysis,  especially  for  tasks  placing  high  demands  on  human  capabilities,  or  for  which  errors  have 
consequences  important  to  safety 

•  Risk-important  human  actions  identified  in  the  HRA  and  PRA  that  are  affected  by  the  upgrade 

•  Initiating  events  considered  in  the  emergency  operating  procedures  that  may  be  affected  by  the  upgrade. 

Integration 

Modifications  should  be  integrated  across  the  full  set  of  procedures;  that  is,  modifications  made  to  particular  parts  of  the 
procedures  should  not  conflict  or  be  inconsistent  with  other  parts,  such  that  their  overall  use  is  impaired. 

Applicability  to  Temporary  HSI  and  Plant  System  Configurations 

Procedures  should  be  developed  for  all  configurations  of  plant  systems  and  HSI  components  with  which  plant  personnel 
interact.  Sometimes  upgrades  are  implemented  over  time,  resulting  in  temporary  configurations  of  plant  systems  or  HSI 
components.  For  example,  a  control  system  may  be  installed,  but  not  all  of  its  operating  modes  (e.g.,  higher-level  automatic 
modes)  may  be  available  initially.  In  addition,  control  or  display  devices  with  different  characteristics  than  the  final  design 
may  be  temporarily  installed.  Thus,  a  display  device  may  have  a  scale  that  is  labeled  differently,  or  it  may  present  a 
variable  that  will  only  be  monitored  during  the  system’s  temporary  configuration  (e.g.,  a  special  radiation  detector 
temporarily  installed  near  a  piece  of  plant  equipment.)  The  procedures  should  accurately  reflect  the  characteristics  of  these 
temporary  configurations  and  the  required  personnel  tasks. 

Evaluation 


The  procedures  should  be  evaluated  to  ensure  their  adequate  content,  format,  and  integration.  The  degree  of  evaluation 
should  be  consistent  with  the  degree  of  change  to  personnel  tasks  and  the  safety  significance  of  those  changes.  If  the 
upgrade  substantially  changes  tasks  that  are  significant  to  plant  safety,  then  verification  and  validation  should  ensure  the 
acceptability  of  the  procedures.  Validation,  described  in  Section  5.4,  should  ascertain  that  they  correctly  reflect  the 
characteristics  of  the  upgraded  plant  and  can  be  carried  out  effectively  to  restore  the  plant’s  state.  Additional  documents 
with  guidance  for  developing  procedures  are  cited  in  Element  8,  Procedure  Development,  NUREG-071 1. 

5.3.3  Training  Development 

Training,  described  in  Element  9  (Section  10)  NUREG-071 1,  should  insure  that  personnel  acquire  the  required  knowledge 
and  skills;  it  is  an  important  factor  in  ensuring  the  safe,  reliable  operation  of  nuclear  power  plants.  Previous  sections  of  this 
report  described  new  demands  that  may  be  imposed  on  operators  by  upgrades  to  systems  and  the  HSI.  These  demands  stem 
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from  changes  in  the  operators’  functions  and  tasks.  This  section  discusses  HFE  considerations  of  establishing  training  for 
upgrades,  including  the  relationship  of  training  development  to  other  HFE  activities.  The  objective  of  this  review  is  to  be 
certain  that  the  training  programs  adequately  address  using  the  upgrades  within  the  context  of  plant  operations.  This 
review  should  also  ensure  that  the  process  used  to  develop  training  is  based  on  the  systematic  analysis  of  job  and  task 
requirements.  Element  9  of  NUREG-071 1  recommends  developing  training  in  coordination  with  the  other  elements  of  the 
HFE  design  process  because  this  can  provide  a  valuable  understanding  of  what  is  required  of  personnel.  For  an  upgrade, 
the  training  program  should  include  all  tasks  that  are  affected  by  it  or  its  interactions  with  the  rest  of  the  plant. 

5.3.3. 1  Determining  Relevant  Knowledge  and  Skills 

The  systematic  approach  to  training,  described  by  NUREG-071 1,  is  a  top-down  process  by  which  programs  are  developed  to 
address  specific  objectives  related  to  the  trainee’s  job.  It  consists  of  five  steps:  (1)  a  systematic  analysis  of  jobs,  (2)  the 
derivation  of  learning  objectives  from  analyses  describing  the  performance  desired  after  training,  (3)  design  and 
implementation  of  training,  (4)  evaluation  of  mastery  of  knowledge,  and,  (5)  evaluation  and  revision  of  the  training.  The 
type  of  performance  desired  after  training  is  often  described  in  terms  of  knowledge,  skills,  and  abilities  (KSA)  (Prien,  1977; 
cited  in  Goldstein,  1987): 

•  Knowledge  -  An  organized  body  of  knowledge,  usually  factual  or  procedural,  whose  application  allows  adequate 
job  performance.  It  is  the  foundation  upon  which  abilities  and  skills  are  built.  However,  possession  of  knowledge 
does  not  insure  that  it  will  be  used. 

•  Skills  -  The  capability  to  perform  a  job  easily  and  precisely.  The  term  is  often  applied  to  both  psychomotor  and 
cognitive  activities.  For  example,  a  psychomotor  skill  may  entail  precisely  operating  a  control  device.  Cognitive 
skill  may  entail  using  strategies  for  rapidly  finding  and  accessing  information  from  the  HSI,  as  discussed  in 
Section  4.3.2.  When  a  skill  is  specified,  a  standard  of  performrmce  often  is  implied. 

•  Abilities  -  Usually  applied  to  cognitive  capabilities  necessary  to  perform  a  job.  Abilities  usually  require  applying 
some  knowledge. 

For  NPPs,  abilities  are  generally  considered  when  selecting  personnel.  Training  usually  focuses  on  gaining  knowledge  and 
skills. 

Identifying  the  knowledge  and  skills  needed  for  complex  human-machine  systems,  such  as  NPP  upgrades,  is  not  always 
easy.  Mumaw  et  al.  (1994)  state  that  while  the  systems  approach  to  training  (SAT)  has  dominated  the  development  of 
training  systems  in  many  domains  due  to  its  standardized  methodology  for  setting  up  comprehensive  and  manageable 
programs,  it  is  most  easily  applied  to  procedural  tasks  in  which  discrete  tasks  are  readily  identified  and  described.  SAT 
typically  is  less  effective  in  the  development  of  instructional  materials  for  less-structured  jobs  that  rely  extensively  on 
cognitive  skills.  SAT  encourages  the  use  of  task  analyses  to  identify  tasks  and  their  associated  physical  and  cognitive  skills. 
However,  it  lacks  techniques  for  analyzing  the  components  of  cognitive  skills  critical  to  successful  performance.  Thus, 
while  critical  decision-making  skills  may  be  identified,  the  SAT  processes  provide  little  support  for  defining  their  important 
elements  or  determining  effective  ways  for  acquiring  these  skills. 

Others  concur  about  the  difficulty  of  defining  knowledge  and  skill  requirements  for  complex  human-machine  systems. 
Sanquist  et  al.  (1995)  state  that  in  human-machine  systems  that  featured  traditional  HSIs,  training  needs  were  often 
specified  by  observing  and  describing  operator  actions.  However,  for  modem  automated  systems,  in  which  greater  demands 
are  placed  on  the  operators’  cognitive  capabilities,  they  consider  that  greater  emphasis  must  be  placed  on  analyzing  aspects 
of  judgement  and  decision  making,  some  of  which  are  not  easily  observed. 
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Experiences  in  the  nuclear  power  industry  and  other  process-control  industries  suggest  that  personnel  are  not  always 
appropriately  trained  for  new  technologies  because  the  analyses  of  training  needs  were  inadequate.  The  following  are  two 
examples.  In  the  first  case,  a  NPP  operator  was  insufficiently  trained  in  the  skills  required  to  operate  a  new  digital 
feedwater  control  system  and,  observing  fluctuations  in  steam  generator  level  during  power  ascension,  incorrectly  concluded 
that  the  full-range,  digital  feedwater  control  system  was  malfunctioning.  The  operator  assumed  manual  control  of  the 
system  and  tried  to  “bump”  open  the  feedwater  valve  with  a  series  of  short  intermittent  key  presses.  However,  the  operator 
was  unaware  that  each  press  corresponded  to  only  about  0.1%  demand,  and  so  the  series  translated  into  negligible  changes 
in  valve  position.  Consequently,  the  plant  tripped  on  low  steam  generator  level. 

The  second  case  involves  training  given  to  operators  in  a  chemical  plant,  which  had  just  installed  its  first  major  digital 
upgrade  to  the  production  system.  The  HSI  for  this  plant  originally  consisted  of  traditional  hardwired  analog 
instrumentation.  When  the  digital  upgrade  was  installed,  management  was  concerned  about  operators’  ability  to  interact 
with  a  new  HSI,  which  featured  CRT-based  displays  and  soft  controls.  Their  training  program  focused  on  interface- 
management  skills,  such  as  using  on-screen  pointers,  and  navigating  display  systems.  However,  the  upgrade  extended 
beyond  the  HSI  and  included  the  control  system,  changing  its  structure  and  behavior.  These  changes  were  not  given  as 
much  attention  in  the  training  program  as  was  interface  management.  While  the  operators  were  generally  able  to  operate 
the  interfaces,  they  lacked  knowledge  and  skills  related  to  the  control-system  changes  and  therefore  had  difficulty  operating 
the  plant  after  this  upgrade.  When  the  second  phase  of  the  upgrade  was  planned,  greater  emphasis  was  placed  on  the 
plant’s  control  characteristics  (O’Hara,  Stubler,  and  Higgins,  1996). 

These  examples  indicate  the  failure  to  consider  all  aspects  of  human  performance  when  developing  training  programs  for 
plant  upgrades.  In  the  first  case,  the  operator  was  inadequately  trained  in  using  the  interface.  (The  problems  were  further 
complicated  by  a  deficiency  in  the  design  of  an  associated  indicator.)  In  the  second  case,  operators  were  trained  in  using  the 
interfaces  but  the  training  did  not  adequately  address  the  changes  in  plant  behavior  resulting  from  modifications  to  systems. 
Thus,  if  the  necessary  dimensions  of  personnel  performance  are  not  recognized  when  analyzing  training  needs,  the  resulting 
training  program  may  have  deficiencies  that  may  not  be  recognized  until  an  incident  occurs.  This  may  be  important  to 
plant  safety. 

Table  5.3  shows  some  categories  of  personnel  performance  that  should  be  considered  when  defining  training  needs  for  an 
upgrade.  Three  topics  are  identified  in  the  first  column:  the  plant,  the  HSI,  and  plant  personnel.  The  first  row,  “Plant 
Interactions,”  covers  changes  to  plant  systems  and  equipment,  and  relates  to  the  primary  tasks  of  plant  personnel,  as 
described  in  Section  4.2.  These  include  the  generic  cognitive  tasks:  monitoring  and  detection,  situation  awareness,  response 
planning,  and  response  implementation.  The  second  column  in  this  row  addresses  the  types  of  knowledge  that  personnel 
may  need  for  these  modified  systems  and  equipment,  such  as  how  they  are  structured,  how  they  function,  how  they  may  fail, 
and  how  they  behave  under  various  conditions.  The  third  column  has  specific  skills  personnel  require  for  their  primary 
tasks  that  may  include  perception  skills  for  monitoring  and  detection;  processing  skills  for  recognizing  patterns,  applying 
heuristics,  generating  hypotheses,  making  decisions;  and  execution  skills  for  manipulating  systems  and  equipment. 

The  next  row,  “HSI  Interactions,”  describes  changes  that  may  be  made  to  the  controls  and  displays  of  the  HSI  in  the 
upgrade.  This  topic  was  a  primary  focus  of  NRC  research  directed  toward  HSIs,  especially  hybrid  ones.  Considerations  in 
this  row  relate  to  interface  management  tasks,  described  as  secondary  tasks  in  Section  4.2.  The  second  column  addresses 
the  specific  categories  of  knowledge  people  may  need  for  the  HSI.  Examples  include  understanding  its  display  structure, 
navigation  paths,  input  methods,  and  failure  modes.  It  may  also  include  knowledge  about  the  types  of  errors  that  may  be 
made  during  interactions  and  the  types  of  strategies  for  recovery. 
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Table  5.3  Some  Knowledge  and  Skill  Dimensions  for  Assessing  Training  Needs 


Topic 

Knowledge 

Skill 

Plant 

Interactions 

Understanding  of  plant  processes, 
systems,  operational  constraints, 
and  failure  modes 

Skills  associated  with  monitoring 
and  detection,  situation  awareness, 
response  planning,  and  response 
implementation 

HSI 

Interactions 

Understanding  of  HSI  structure, 
functions,  failure  modes,  and 
interface-management  tasks 
(actions,  errors,  and  recovery 
strategies) 

Skills  associated  with  interface- 
management  tasks 

Personnel 

Interactions 

(In  the  CR  and  in  the  Plant) 

Understanding  of  information 
requirements  of  others,  how 
actions  must  be  coordinated  with 
others,  policies  and  constraints  on 
the  crew’s  interactions 

Skills  associated  with  crew’s 
interactions  (i.e.,  teamwork) 

The  final  row,  “Personnel  Interactions,”  addresses  possible  changes  in  the  interactions  between  personnel  after  an  upgrade. 

It  includes  interactions  between  personnel  in  the  CR,  between  the  CR  and  the  rest  of  the  plant,  and  between  personnel 
located  in  the  plant.  This  category  was  described  in  the  discussions  of  crew  performance  and  team  skills  (Sections  4.4.2  and 
5.2.2).  Its  primary  focus  is  interactions  involving  CR  personnel  that  may  be  important  to  plant  safety.  The  second  column 
addresses  the  types  of  knowledge  that  support  coordination  of  information  and  actions  between  personnel,  such  as  a 
common  understanding  of  the  plant  and  how  it  affects  everybody’s  roles  (shared  mental  model),  of  the  information 
requirements  and  tasks  of  others,  and  of  the  plant’s  policies  and  practices  (e.g.,  administrative  procedures)  related  to 
people’s  interactions.  The  third  column  addresses  skills  that  are  important  to  teamwork.  Examples  may  include  perceptual 
skills  (e.g.,  maintaining  awareness  of  CR  activities),  processing  skills  (e.g.,  identifying  when  coordination  is  needed),  and 
other  communicating  and  coordinating  skills. 

In  studying  training  requirements  for  severe  accident  management,  Mumaw  et  al.  (1994)  developed  a  detailed 
categorization  of  cognitive  skills  that  extends  beyond  the  knowledge  and  skill  distinction  in  Table  5.3.  Their  categorization 
is  based  on  a  framework  by  Newell  and  Simon  (1972,  cited  by  Mumaw  et  al.,  1994)  in  which  behavior  is  directed  by  a  series 
of  comparisons  between  the  current  state  and  the  desired  state  of  the  system.  For  example,  an  operator  faced  with  a  variable 
that  deviates  from  a  desired  value  must  determine  another  appropriate  path  to  the  goal  using  available  means.  The 
problem-solver’s  task  is  to  continually  modify  the  problem  (e.g.,  the  variable)  until  it  is  identical  with  the  goal  (e.g.,  the 
desired  value).  The  actual  and  goal  value  may  be  compared  periodically  to  evaluate  progress.  Mumaw  et  al.  propose  that 
when  developing  training  programs  for  complex,  cognitively  demanding  tasks,  analysts  should  first  explicitly  model  the 
characteristics  of  highly  skilled  ijerformance.  They  identified  five  elements:  task-relevant  knowledge,  mental 
representation,  rules  for  processing  knowledge,  domain  goals  and  subgoals,  and  strategies: 

Task-Relevant  Knowledge -This  refers  to  knowledge  about  the  characteristics  of  the  plant,  HSI,  crew,  and  performance 
expectations  relevant  to  a  specific  task.  Types  of  knowledge  that  may  apply  to  particular  tasks  include  thermodynamic 
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theory,  plant  systems  (e.g.,  their  structure,  interconnections,  and  operating  characteristics),  transient  phenomena,  the  logic 
underlying  procedures,  plant-specific  facts,  and  plant  policies. 

Mental  Representation  —  An  initial  step  in  problem  solving  is  to  develop  mental  representations  of  the  current  state  of  the 
system  and  the  goal.  These  skills  allow  personnel  to  group  information  from  the  task  (e.g.,  indications  from  alarms  and 
displays)  into  meaningful  patterns  relating  to  the  user’s  understanding  of  the  plant  (situation  model).  They  are  critical  for 
monitoring  and  interpreting  data  about  the  plant.  As  Sections  4.3.2  and  4.3.3  describe,  proficient  operators  develop  skills 
for  extracting  information  from  their  environment  based  on  their  ability  to  recognize  meaningful  patterns  and  to  understand 
functional  relationships  between  objects  and  events.  These  skills  are  important  because  they  allow  individuals  to  manage 
their  mental  resources  effectively. 

Rules  for  Processing  Knowledge  -  Skilled  problem  solvers  often  apply  rules  to  modify  the  representation  of  the  problem  so 
that  it  more  closely  matches  the  goal.  Thereby,  the  problem  solver  can  address  the  problem  in  a  series  of  stages,  or  restate  it 
in  a  form  that  can  be  more  easily  solved.  A  single  rule  may  be  applied  to  simple  problems.  More  complicated  ones  may 
require  multiple  transformations  involving  many  rules.  For  example,  determining  whether  the  reactor  core  is  adequately 
cooled  during  an  emergency  may  require  the  operator  to  assess  the  status  of  several  systems  and  evaluate  the  performance  of 
multiple  flow  paths  and  heat  sinks.  Each  time  a  processing  rule  is  applied,  a  particular  subgoal  may  be  resolved  and  the 
problem  state  brought  closer  to  the  goal. 

Domain  Goals  and  Subgoals  -  Applying  knowledge-processing  rules  to  transform  the  problem  usually  requires  an 
understanding  of  the  relationships  between  a  string  of  subgoals  to  approach  the  ultimate  goal.  By  understanding  such 
relationships,  personnel  can  apply  knowledge  processing  rules  effectively. 

Strategies  -  Mumaw  et  al.  describe  mental  strategies  as  the  ability  to  string  together  individual  knowledge-processing  rules 
to  achieve  a  task-relevant  goal.  For  example,  in  electronics  troubleshooting,  skilled  technicians  use  a  “split-half’  strategy  to 
isolate  faults  in  complex  circuits  (Stubler,  Higgins,  and  Kramer,  2000).  If  a  circuit  has  a  test  point  that  produces  a  bad 
signal  and  another  that  produces  a  good  signal,  the  technician  will  test  a  point  half  way  between  them  to  determine  whether 
the  fault  lies  upstream  or  downstream.  By  applying  this  strategy  repeatedly  on  the  bad  half,  the  problem  is  transformed  to 
include  progressively  smaller  portions  of  the  circuit,  allowing  the  technician  to  locate  the  fault  without  testing  each  point  of 
the  circuit  individually. 

Thus,  by  analyzing  the  tasks  of  skilled  personnel  and  expressing  them  in  terms  of  task-relevant  knowledge,  mental 
representation,  rules  for  processing  knowledge,  domain  goals  and  subgoals,  and  strategies,  analysts  can  identify  training 
objectives  that  address  the  knowledge  and  cognitive  skills  required  for  skilled  performance. 

5.3.3.2  Approaches  to  Training  Cognitive  Skills 

There  are  numerous  techniques  and  methods  for  training  cognitive  skills.  In  examining  methods  for  training  of  NPP 
personnel,  Mumaw  et  al.  (1994)  identified  a  taxonomy  of  19  methods  to  address  the  categories  of  cognitive  skills  described 
above,  and  grouped  them  into  seven  training  goals.  Mumaw  et  al.  acknowledge  that  these  categories  are  arbitrary  since  a 
single  training  method  may  cover  more  than  one  goal  depending  upon  how  it  is  carried  out.  However,  this  categorization  is 
a  useful  way  of  matching  established  training  methods  to  the  dimensions  of  cognitive  skill.  Six  of  these  seven  categories  are 
applicable  to  training  programs  directed  toward  upgrades;  five  goals  are  described  below.  The  sixth  training  goal,  the 
teaching  team’s  skills,  is  discussed  in  Section  5.3.3.6.  Mumaw  et  al.  (1994)  give  a  thorough  discussion  of  all  seven 
categories. 
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Teaching  Knowledge  -  The  primary  concern  here  is  knowledge  that  personnel  should  be  able  to  access  from  their  memory, 
rather  than  from  external  sources  such  as  procedure  documents  and  displays.  For  upgrades,  this  training  should  include 
knowledge  about  those  changes  made  to  systems  or  the  HSI  upgrades  that  relate  to  personnel  tasks.  This  may  include  the 
unique  characteristics  of  the  upgrades,  their  relationships  to  the  rest  of  the  plant,  changes  in  requirements  for  coordination 
(e.g.,  when  and  how  CR  operators  must  coordinate  their  actions),  and  new  expectations  about  personnel  or  plant 
performance  that  stem  from  the  upgrades. 

Mumaw  et  al.  recommend  that  knowledge  training  address  two  types  of  knowledge  failures:  knowledge  that  is  not  tied  to 
task  performance  (i.e.,  it  is  inert)  and  knowledge  that  is  forgotten  when  needed.  Inert  knowledge  (Bereiter  and 
Scardamalia,  1985,  cited  in  Mumaw  et  al.  1994;  Mann  and  Hammer,  1986)  can  be  applied  in  a  training  setting  by  the 
trainee  but  cannot  be  used  effectively  in  the  task  setting.  It  fails  to  become  part  of  the  trainee’s  usable  store  of  knowledge 
because  the  it  is  not  tied  to  the  context  of  task  performance.  Mumaw  et  al.  recommend  defining  the  job’s  context  during 
training  and  representing  the  task  meaningfully;  this  links  the  knowledge  to  the  context  of  the  Job.  Mann  and  Hammer 
(1986)  state  that  to  avoid  inert  knowledge,  theoretical  training  should  be  integrated  with  training  in  the  use  of  procedures. 
Mumaw  et  al.  describe  six  training  methods  which  address  both  types  of  knowledge  failures.  Considerations  about 
determining  the  appropriate  level  of  conceptual  knowledge  to  be  provided  by  training  is  discussed  in  Section  5.3. 3.4. 

Teaching  Knowledge  Representation  -  Simply  having  task-relevant  knowledge  is  not  sufficient  for  skilled  performance. 
Personnel  must  learn  to  organize  information  extracted  from  the  work  environment  so  it  can  be  used  effectively  by  the 
available  mental  resources.  This  training  should  include  knowledge  and  skills  required  for  understanding  the  structure  and 
behavior  of  the  plant  system  or  HSI  upgrades.  It  is  particularly  important  that  people  can  develop  adequate  mental  models 
of  automated  components  (e.g.,  new  control  modes)  and  the  HSI  (e.g.,  features  of  the  display  system  that  manipulate  or 
manage  the  presentation  of  information  to  users). 

While  users  may  develop  these  knowledge-representation  skills  on  their  own  after  extensive  experience  with  an  upgrade,  it 
is  preferable  to  train  them  in  these  skills  to  avoid  initial  decrements  in  performance  that  may  occur  before  they  have  enough 
experience  to  refine  these  skills  by  themselves.  Knowledge-representation  training  for  an  upgrade  should  focus  on  two 
areas.  First,  people  should  be  trained  in  recognizing  perceptual  patterns  in  the  new  HSI,  and  taught  how  to  apply  existing 
perceptual  skills,  developed  for  the  old  HSI,  to  the  upgraded  one.  If  existing  skills  are  not  applicable,  then  new  skills  should 
be  taught.  Second,  the  knowledge-representation  training  should  discuss  mental  models  to  give  an  adequate  understanding 
of  the  structure  and  behavior  of  the  upgrades. 

Teaching  Rules  Applied  to  Decision  A/aA/wg-  When  developing  this  training  for  upgrades,  analysts  should  identify  the 
applicable  rules.  Thus,  for  system  upgrades,  the  knowledge-processing  rules  may  pertain  to  interpreting  symptoms 
associated  with  malfunctions.  For  example,  automated  systems  sometimes  can  compensate  for  the  performance  of  a  slowly 
degrading  plant  system  and  mask  the  symptoms  of  the  malfunctions  (O’Hara,  Stubler,  and  Higgins,  1996).  When  new 
levels  of  automation  are  introduced,  operators  and  maintenance  personnel  may  need  to  learn  new  rules  for  interpreting  the 
plant’s  symptoms  and  detecting  these  malfunctions.  For  HSI  upgrades,  such  as  a  computer-based  display  system,  personnel 
may  need  to  learn  rules  for  interpreting  the  information  from  the  new  HSI  components  or  detecting  failures.  They  may 
need  to  learn  new  rules  for  interacting  with  new  types  of  information,  or  new  control  and  display  capabilities  provided  via 
graphical  user  interfaces. 

Training  that  is  intended  to  teach  rules  for  decision  making  should  seek  to  eliminate  incorrect  rules,  called  “buggy  rules” 
(Brown  and  Burton,  cited  in  Mumaw  et  al.,  1994).  Skilled  personnel  develop  rules  to  support  them  in  making  decisions 
(e.g.,  alarm  B  always  occurs  shortly  after  alarm  A).  However,  these  rules  may  not  invariably  reflect  the  true  behavior  of  the 
plant,  or  may  be  overgeneralized  to  non-applicable  situations.  Buggy  rules  may  also  include  ones  that  worked  correctly 
before  the  upgrade  but  are  no  longer  appropriate. 
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Teaching  Strategies,  Goals,  and  Subgoals  -  This  training  addresses  problem  solving  and  decision  making  which  entail 
decomposing  a  problem  into  an  organized  collection  of  goals  and  subgoals,  and  then  identifying  specific  rules  that  can  be 
applied  to  each.  It  may  involve  using  strategies  -  a  sequence  of  rules  used  to  achieve  a  subgoal  or  goal.  Together,  the  use 
of  strategies,  goals,  and  subgoals  is  essential  to  response  planning. 

This  training  may  be  particularly  important  when  plant  systems  are  upgraded.  Such  changes  may  affect  the  relationships 
between  goals  and  subgoals,  and  the  paths  available  for  controlling  the  plant’s  state.  In  the  upgraded  plant,  some  subgoals 
may  no  longer  be  relevant  to  higher-level  goals  and  personnel  may  need  to  learn  new  strategies.  The  introduction  of  more 
advanced  levels  of  automation  may  change  the  considerations  involved  in  selecting  response  paths.  For  example,  a  new 
automation  system  may  simultaneously  control  a  large  number  of  plant  variables,  and  die  control  actions  may  involve  a 
different  set  of  side  effects  than  existed  before  the  upgrade.  For  example,  a  larger  number  of  plant  systems  may  be  affected 
by  a  decision  to  place  this  system  in  a  manual-control  mode. 

Changes  in  the  HSI  may  affect  the  strategies  that  personnel  can  use  for  accessing  information  about  the  plant  or  carrying 
out  control  actions.  Older  strategies  for  scanning  spatially  distributed  alarms  and  displays  may  be  inappropriate  in  an 
upgraded  HSI  with  computer-based  displays.  From  a  review  of  eight  HSI  upgrades,  Heslinga  and  Herbert  (1993)  concluded 
that  where  conventional  HSI  equipment  was  still  available  after  the  upgrade,  operators  tended  to  return  to  these  well-known 
information  sources.  Operators  tended  to  develop  strategies  for  finding  information  quickly  from  conventional  HSI 
equipment,  such  as  control  panels  and  displays,  because  their  arrangement  provided  overviews  of  status  and  they  were  more 
directly  accessible.  By  contrast,  obtaining  an  overview  and  accessing  controls  and  displays  were  considered  to  be  more 
difficult  in  hierarchically  structured  VDU  display  systems. 

In  reviewing  the  implementation  of  a  computer-based  display  system  in  a  NPP,  Roth  and  O’Hara  (1998)  found  that  training 
in  strategies  for  using  the  HSI  was  very  important.  The  relevant  findings  are  summarized  below. 

Because  training  for  NPP  operators  tends  to  focus  on  controlling  the  plant,  adequate  attention  may  not  be  given  to 
effectively  using  HSI  capabilities  during  events.  In  this  study,  the  utility  had  focused  little  attention  on  such  training. 

There  had  been  little  discussion  on  how  the  HSI  components  were  used,  or  how  they  could  be  used  better;  this  was  left  to 
individual  crew  members  to  decide.  Consequently,  there  was  much  individual  variability  within  and  between  crews  in  the 
extent  to  which  and  how  they  used  the  different  systems  (e.g.,  the  graphic  display  system).  For  example,  the  computer- 
based  display  system,  especially  the  computer-based  procedure  (CBP)  component,  reduced  the  demands  on  the  attentional 
resources  of  operators  under  some  conditions,  but  not  all  operators  may  take  advantage  of  this  benefit.  One  shift  engineer 
pointed  out  that  the  board  operators  in  his  crew  took  advantage  of  their  freed  attentional  resources,  and,  as  a  result,  kept 
better  track  of  the  event.  However,  he  was  unsure  of  whether  other  board  operators  also  did  so.  He  felt  there  may  be  a  need 
to  specifically  train  operators  to  take  advantage  of  the  attentional  resources  that  were  freed  by  introducing  the  CBP. 

The  lack  of  explicit  training  in  using  the  HSI  systems  reflects  variously  (a)  an  acceptance  or  expectation  that  operators  will 
develop  familiarity  with  the  various  features  of  the  systems  informally  on  their  own  (i.e.,  that  there  is  no  need  to  teach  this 
knowledge  and  skill),  and  (b)  an  acceptance  or  expectation  that  different  individuals  will  tailor  the  systems  to  their  own 
personal  style  (e.g.,  that  they  will  put  up  different  graphic  displays  or  different  collections  of  windows  on  their  VDUs). 
There  was  a  similar  lack  of  formal  training  in  fossil-fuel  power  plants  and  chemical  plants  visited  as  part  of  this  project 
(O’Hara,  Stubler,  and  Higgins,  1996).  Operators  developed  informal  practices  for  arranging  particular  displays  in  these 
computer-based  CRs.  Arrangements  varied  between  control  rooms  even  in  multiple-unit  plants  where  the  systems  and  HSIs 
were  nearly  identical.  In  one  chemical  plant,  operators  failed  to  detect  the  early  stages  of  a  serious  cascading  failure  of  a 
system  because  their  arrangement  of  display  pages  did  not  allow  them  to  immediately  understand  the  problem  (O’Hara, 
Stubler,  and  Nasta,  1997).  While  it  is  important  to  guard  against  over-regimentation  and  to  allow  operators  flexibility  in 
tailoring  their  VDU  work  space  to  their  particular  needs,  such  flexibility  increases  the  demands  on  operators  to  engage  in 
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tasks  not  directly  related  to  process  control.  Because  operators  are  often  reluctant  to  perform  such  tasks  in  high-workload 
situations  (O’Hara,  Stubler,  and  Nasta,  1997),  training  may  improve  performance  during  these  situations.  Because 
empirical  studies  are  lacking,  it  is  unclear  how  this  training  should  be  conducted,  or  the  extent  to  which  performance  may 
be  improved.  The  training  recommendations  of  Mumaw  et  al.  (1994)  provide  some  initial  direction. 

In  addition  to  requiring  personnel  to  learn  new  strategies,  introducing  new  technology  may  create  different  demands  for 
maintaining  the  old  strategies.  This  is  especially  true  when  an  old  HSI  technology  is  maintained  as  a  backup  for  use  when 
the  new  HSI  technology  is  inoperable  or  unavailable.  In  this  case,  personnel  must  maintain  their  skills  for  both  systems  and 
develop  skills  to  transfer  smoothly  from  one  to  the  other.  An  important  example  is  making  the  transition  between 
emergency  operating  procedures  (EOPs)  that  are  in  computer-based  form  to  EOPs  that  are  in  paper-based  form  when  the 
CBP  system  fails  (O’Hara,  Higgins,  Stubler,  and  Kramer,  2000).  Paper-based  procedures  require  the  operator  to  mentally 
track  many  things,  including  transitions  between  EOPs,  cautions  and  warnings  that  apply  to  multiple  steps,  and  steps  that 
may  require  time  for  completion  (e.g.,  monitoring  a  variable  until  a  particular  value  is  reached).  CBPs  may  require  a 
different  set  of  skills  if  these  tasks  are  automated.  For  example,  operators  must  develop  skills  for  interacting  with  a 
computer-based  interface.  In  addition,  CBPs  may  impose  new  demands  such  as  the  need  to  manage  the  automation  and 
ensure  its  proper  operation.  In  a  study  of  the  introduction  of  CBPs  into  a  NPP  (Roth  and  O’Hara,  1998),  operators  stated 
that  it  will  be  important  to  be  trained  in  using  paper-based  procedures  so  they  do  not  lose  their  valuable  skills  in  this 
medium.  However,  this  concern  is  not  limited  to  CBPs.  Any  time  an  older  technology  is  kept  as  a  backup  to  a  newer  HSI 
technology,  personnel  may  be  required  to  maintain  separate  sets  of  skills;  this  is  likely  to  increase  training  demands. 

Mumaw  et  al.  (1994)  state  that  strategies,  goals,  and  subgoals  training  should  support  personnel  in 

•  Determining  the  goals  that  are  most  relevant  to  the  current  plant  state 

•  Selecting  paths  for  achieving  those  goals 

•  Ensuring  that  actions  specified  by  procedures  are  consistent  with  those  goals  and  paths. 

Teaching  Management  of  Mental  Resources  -  Training  personnel  in  managing  their  mental  resources  is  important  both  for 
increasing  the  effectiveness  of  personnel  training  programs,  and  for  enhancing  performance  on  the  job.  This  training  is 
particularly  important  when  the  HSI  has  been  upgraded.  It  should  address  changes  in  the  ways  that  the  HSI  presents 
information  and  control  capabilities,  and  the  actions  that  personnel  may  use  to  interact  with  it.  Training  should  focus  on 
reducing  the  amount  of  conscious  attention  required  for  tasks  such  as  searching  for  and  retrieving  information  from  the 
display  system  and  for  routine  control  actions.  One  such  method,  automaticity  training,  is  discussed  further  in  Section 
5.3.3.4. 

5.3.3.3  Training  for  Conceptual  Knowledge 

As  new  technologies  are  implemented  in  NPPs,  such  as  increased  automation  in  plant  systems  and  advanced  HSI 
capabilities,  greater  demands  may  be  placed  on  personnel  for  understanding  their  structure  and  operation.  Most  approaches 
to  training  development  advocate  conducting  training  within  the  context  of  the  individual’s  job  by  linking  knowledge 
requirements  to  specific  tasks.  However,  determining  the  appropriate  level  of  conceptual  knowledge  (e.g.,  facts  about  the 
structure  and  behavior  of  the  plant)  poses  some  concerns.  The  following  describes  some  considerations  identified  from  HFE 
literature  and  interviews  with  personnel. 

Interviews  with  personnel  from  nuclear,  fossil-fuel,  and  chemical  plants,  conducted  in  this  project  indicated  that  operators 
who  are  superior  performers  have  a  greater  understanding  of  what  makes  the  plant  work  and  how  it  will  behave  (O’Hara, 
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Stubler,  and  Higgins,  1996).  While  less  knowledgeable  operators  view  their  tasks  as  a  series  of  control  manipulations, 
superior  operators  have  a  greater  understanding  of  why  they  are  performing  them.  Furthermore,  the  importance  of  training 
operators  in  conceptual  knowledge  was  indicated  by  empirical  studies.  For  example,  troubleshooting  by  professional 
engineering  officers  in  the  engine  CR  of  a  supertanker  was  studied  in  a  high-fidelity  CR  simulator  (van  Eekhout  and  Rouse, 
1981;  cited  in  Rouse,  1990).  It  was  found  that  errors  associated  with  inappropriately  identifying  failures  had  a  high 
correlation  with  a  lack  of  knowledge  about  the  functioning  of  the  basic  system  and  its  automatic  controllers.  The  errors 
tended  toward  mistakes  (errors  of  intention)  rather  slips  (errors  of  execution).  That  is,  the  officers  planned  inappropriate 
actions  based  on  their  inadequate  or  incorrect  knowledge  of  the  system.  Errors  in  diagnosing  failures  of  automatic 
controllers  were  often  associated  with  a  lack  of  understanding  of  their  failure  modes  which  then  affected  the  officers’ 
abilities  to  interpret  the  symptoms  of  failure.  Officers  were  misled  by  the  cues  produced  by  these  failures,  drew  incorrect 
conclusions,  and  then  proceeded  to  act  on  them.  This  phenomenon  was  reportedly  widespread  in  a  series  of  simulations. 

However,  interviews  with  NPP  instructors  suggested  that  training  oriented  toward  conceptual  knowledge  rather  than 
procedural  knowledge  (e.g.,  how  to  do  something)  can  cause  operators  to  overestimate  their  ability  to  diagnose  and  correct 
plant  anomalies.  For  example,  operators  may  deviate  from  procedures  because  they  feel  they  understand  where  the 
procedure  is  heading  and  what  is  needed  to  correct  the  problem.  Deviating  from  the  procedures  can  create  new  problems. 

In  other  cases,  operators  who  feel  they  understand  a  plant  problem  may  attempt  to  act  alone  in  resolving  it,  rather  than 
obtaining  support  from  specialists.  Difficulties  can  occur  when  the  operator’s  course  of  action  does  not  fully  account  for  the 
range  of  factors  that  the  specialist  may  consider  (O’Hara,  Stubler,  and  Higgins,  1996). 

Further  support  for  the  notion  that  training  in  conceptual  knowledge  can  allow  operators  to  use  that  knowledge 
inappropriately  was  found  by  Mann  and  Htunmer  (1986)  studying  two  groups  trained  to  operate  a  simulated  generic 
process-control  plant.  One  group  was  provided  with  procedures  for  controlling  the  plant;  the  other  group  also  had 
instruction  in  the  theory  of  the  plant,  such  as  a  description  of  the  physical  principles  of  its  operation.  One  finding  was  that 
the  latter  group  committed  more  blatant  errors  (called  blunders)  and  omitted  more  procedure  steps.  The  blunders  included 
obviously  inappropriate  actions,  such  as  ignoring  procedure  branching,  using  a  procedure  when  the  situation  did  not  call  for 
it,  ignoring  an  action  called  for  by  a  procedure,  and  ignoring  parameter  ranges  specified  by  a  procedure.  The  experimenters 
suggest  that  participants  in  the  procedures  plus  theoretical  instruction  group  may  have  selectively  omitted  or  ignored 
procedures  because  their  theoretical  training  caused  them  to  think  they  could  achieve  better  performance  by  improvising. 
They  concluded  that  procedure-only  training  resulted  in  better  compliance  than  training  on  procedures  plus  theory. 

As  the  plant  systems  and  HSIs  of  NPPs  increase  in  complexity  with  the  introduction  of  new  technologies,  the  importance  of 
questions  about  the  necessary  level  of  conceptual  knowledge  is  likely  to  increase.  Further  research  is  required  to  develop 
more  definitive  guidance. 

5.3.3.4  Training  for  Skill  Automaticity 

Schlager  et  al.  (1989)  describe  the  typical  process  by  which  novices  learn  to  “automate”  certain  actions.  During  a  trainee’s 
initial  introduction  to  a  task,  the  required  actions  are  typically  presented  as  a  set  of  ordered  steps.  Each  step  requires 
attention  and  is  consciously  initiated  by  the  trainee.  At  this  learning  stage,  the  actions  are  stored  as  factual  knowledge  and 
the  trainee  usually  has  little  difficulty  in  describing  each  step.  When  an  entire  set  of  actions  is  linked  to  a  discemable  event, 
the  set  can  be  developed  after  sufficient  practice  into  a  single,  coherent  unit  that  is  executed  “automatically.”  When  the 
event  occurs,  the  individual  can  undertake  the  set  of  actions,  focusing  little  attention  on  the  individual  ones.  This  is 
essentially  a  conversion  of  factual  knowledge  into  procedural  knowledge  (Anderson,  1981;  cited  by  Schlager  et  al.,  1989). 

Automaticity  training  allows  this  process  to  occur  in  the  training  setting  rather  than  on  the  job;  one  purpose  is  to  allow  the 
trainee  to  attain  a  high  degree  of  skill  early  in  the  program.  This  can  free-up  the  cognitive  resources  of  the  trainee  for 
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higher-level  activities,  which  can  make  the  training  program  more  efficient  and,  possibly,  more  effective.  That  is,  the 
trainee  can  focus  on  learning  higher-level  skills,  such  as  problem  solving,  decision  making,  and  coordination  of  concurrent 
activities,  without  being  distracted  by  the  details  of  these  lower  level  actions.  A  second  purpose  of  automaticity  training  is 
to  enhance  performance  on  the  “automated  task.”  Empirical  studies  have  shown  large  improvements  in  the  speed  and 
accuracy  of  perceptual  searches  and  motor  responses  after  automaticity  training  (Schneider,  1985). 

The  following  are  suggestions  for  training  operators  to  achieve  automaticity,  adapted  from  those  offered  by  Schlager  et  al. 
(1989): 


•  Identify  opportunities  for  automaticity  training  by  identifying  user-system  interactions  that  are  performed  many 
times  in  the  same  way. 

•  Identify  the  conditions  of  applicability  for  each  new  user-system  interaction  to  ensure  that  they  are  covered  by 
training. 

•  Identify  user-system  interactions  that  are  likely  to  become  confused,  and  point  them  out  to  the  trainee. 

•  For  each  user-system  interaction,  identify  and  make  explicit  each  step  required. 

•  Provide  sufficient  practice  under  the  appropriate  conditions  so  that  users  can  perform  each  interaction 
automatically. 

User-system  interactions  that  are  likely  to  become  confused  are  ones  with  inconsistencies  between  new  interactions  and  the 
old  ones.  Confusion  may  also  result  from  inconsistencies  between  similar  new  interactions  in  the  new  HSI  (e.g.,  the  new 
interactions  of  an  HSI  upgrade  may  be  inconsistent  with  each  other).  Both  of  these  cases  were  described  in  Section  5.3.1. 

In  these  cases,  trainees  should  be  taught  practices  to  avoid  errors,  and  rules  for  identifying  and  correcting  them. 

Automaticity  training  typically  involves  practicing  the  task  in  isolation  from  other  tasks  over  a  period  until  the  response 
time  and  demands  for  attention  decrease.  This  may  require  the  trainee  to  perform  many  trails  over  days  (Mumaw  et  al., 
1994).  Next,  the  task  should  be  integrated  with  other  tasks  in  dual-  or  multiple-task  training  exercises.  When  automated 
skills  are  integrated  back  into  a  multiple-task  setting,  an  initial  decrement  in  performance  typically  occurs  because  skills 
learned  in  isolation  may  be  learned  inefficiently.  By  practicing  the  skill  when  additional  requirements  for  mental  resources 
are  present,  such  as  a  second  task,  the  trainee  can  learn  needed  strategies  for  sharing  attention  and  memory  resources.  After 
the  skill  has  been  mastered  in  this  setting,  the  trainee  is  ready  to  apply  it  efficiently  in  a  realistic  task  setting. 

When  developing  training  for  automaticity,  it  is  important  to  consider  the  scope  of  the  upgrade  and  the  effect  that  the  initial 
limited  level  of  performance  may  have  on  overall  plant  operations;  it  may  have  little  effect  on  plant  operations  if  the  scope 
of  the  upgrade  is  limited  to  HSI  components  that  are  used  infrequently  and  have  little  associated  safety  significance. 
However,  automaticity  training  may  be  important  if  the  upgrade  includes  components  that  are  significant  to  safety,  or  used 
so  frequently  during  transients  that  poor  performance  may  interfere  with  other  tasks  that  are  important  to  safety.  In  such 
cases,  operators  should  be  trained  in  the  use  of  the  upgrade  to  an  extent  that  they  can  perform  the  necessary  actions  without 
having  to  recall  or  attend  to  the  details  of  the  actions. 

5.3.3.5  Team  Training 

Team  training  should  be  undertaken  when  the  design  of  an  upgrade  significantly  affects  personnel’s  ability  to  interact,  and 
if  these  interactions  are  important  to  the  plant’s  performance  and  safety.  Studies  of  team  performance  in  complex  human- 
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machine  systems  have  found  that  it  may  be  negatively  affected  by  introducing  computer-based  technologies  (Hutchins, 
1990). 

Roth  and  O’Hara  (1998)  reviewed  the  introduction  of  a  computer-based  display  system  into  an  NPP  CR.  One  component  in 
particular,  a  computer-based  procedure  system,  changed  the  ways  in  which  CR  personnel  interacted  when  using  procedures. 
Many  of  the  operators’  comments  focused  on  the  need  for  more  practice  on  crew-interaction  skills,  including  focused 
training  on  their  new  roles  and  responsibilities,  communication,  and  coordination.  Several  participating  crews  pointed  out 
that  training  as  a  crew  will  become  more  important  with  the  new  HSI  systems.  Because  the  operators  now  have  more 
freedom  in  how  they  do  their  jobs,  it  is  more  important  to  understand  how  teammates  perform  their  tasks.  For  example, 
when  performing  procedure-based  scenarios,  different  shift  supervisors  verbalize  the  EOP  steps  to  different  degrees.  With 
the  new  system,  it  is  becoming  more  important  for  operators  to  understand  how  shift  supervisors  do  this,  and  for  the  shift 
supervisors  to  understand  the  information  requirements  of  the  operators.  It  was  concluded  that  shift  supervisors  should  be 
trained  in  how  to  keep  their  crews  informed  as  they  move  through  a  procedure.  In  this  plant,  crews  did  not  always  train  as 
a  unit  if  there  were  scheduling  constraints.  Instead,  operators  sometimes  trained  with  people  from  different  crews. 

Other  findings  by  Roth  and  O’Hara  (1998)  that  relate  to  team  training  include  the  following: 

•  The  study  reinforced  the  notion  that  the  quality  of  decision  making  depends  on  the  ability  of  every  team  member  to 
be  aware  of  important  control  actions  about  to  be  taken,  and  to  evaluate  their  appropriateness  based  on  their  own 
knowledge  and  perspective.  Several  critical  incidents  were  observed  in  which  inappropriate  control  actions, 
suggested  by  the  shift  supervisor  based  on  the  CBP,  were  caught  by  the  board  operators  or  the  shift  engineer  who 
were  accessing  different  sources  of  information. 


•  The  study  further  highlighted  the  fact  that,  in  a  control  room  environment,  key  information  is  distributed  among 
.  crew  members.  Advanced  HSI  systems  increase  the  knowledge  and  understanding  of  individual  crew  members 

through  increased  availability  of  information.  On  the  other  hand,  more  information  can  result  in  increased 
compartmentalization  of  knowledge,  such  that  different  crew  members  are  more  likely  to  possess  different  data. 
This  places  an  increased  premium  on  effective  communication  among  crew  members  to  share  information. 

•  A  useful  technique  for  enhancing  communication  within  a  crew  may  be  to  cross-train  operators  so  that  each  is 
aware  of  the  communication  needs  and  burdens  of  the  other.  For  example,  board  operators  may  be  trained  in  the 
shift  supervisor’s  tasks  of  using  the  CBP.  This  may  help  operators  understand  the  type  of  information  that  the  shift 
supervisor  has  and  the  type  of  information  that  is  needed  from  the  operators. 

•  A  wide  variability  was  observed  among  crews  in  the  extent  and  content  of  their  communication.  The  impact  of 
such  differences  on  individual  and  crew  situation  awareness  and  quality  of  team  decision  making  is  not  well 
understood.  More  research  is  required  on  this  topic  to  provide  better  guidance  for  developing  and  evaluating 
training  programs. 

The  study  of  generic  factors  that  contribute  to  the  successful  performance  of  teams  currently  is  growing.  Models  describing 
the  components  of  teamwork  and  methods  for  measuring  and  enhancing  teamwork  are  evolving  from  research  (Swezey  and 
Llaneras,  1997;  Salas,  Dickinson,  Converse,  and  Tannenbaum,  1992;  Oser,  McCallum,  Salas,  and  Morgan,  1989).  For 
example,  in  a  study  sponsored  by  the  NRC,  a  set  of  six  performance  dimensions  was  identified  for  describing  team 
performance  in  NPP  CRs  (Montgomery  et  al.,  1992,  cited  in  Mumaw  et  al.,  1994):  communication,  task  coordination, 
maintaining  task  focus  in  transitions,  adaptability,  openness  and  participation,  and  team  spirit.  A  similar  set  of 
performance  dimensions  was  identified  for  aviation  crews  (Franz  et  al.,  1990,  cited  in  Mumaw  et  al.,  1994): 
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communication,  situation  awareness,  decision  making,  mission  analysis,  leadership,  adaptability  and  flexibility,  and 
assertiveness. 

Observation  is  a  common  approach  for  exploring  team  skills  in  complex  work  environments.  Analysts  observe  team 
performance  to  identify  behaviors  linked  to  specific  skills.  For  example,  when  a  new  piece  of  information  about  the  plant’s 
status  becomes  available  in  a  CR,  analysts  may  watch  for  indications  that  crew  members  are  communicating  effectively  to 
share  the  new  information  (Mumaw  et  al.,  1994).  An  approach  similar  to  the  following  may  be  used  for  developing  team 
training  programs  that  address  upgrades: 

•  Identify  relevant  upgrade  characteristics  -  The  design  characteristics  of  the  upgrade  that  are  important  to  crew 
performance  should  be  identified  from  the  function  and  task  analyses  performed  in  developing  the  design  of  the 
upgrade. 

•  Identify  relevant  team  skills  -  Team  skills  involved  in  performing  the  crew’s  functions  and  tasks  should  be 
identified. 

•  Identify  specific  behaviors  -  Next,  specific  behaviors  should  be  identified  which  demonstrate  those  skills.  These 
may  be  specific  tasks  involving  use  of  the  HSl  by  many  team  members.  For  example,  the  skill  needed  for 
communicating  the  plant’s  status  to  the  primary  decision  maker  occurs  at  many  points  in  plant  operations,  such  as 
when  operators  periodically  report  core-reactivity  values  during  a  startup. 

•  Develop  training  scenarios  -  A  set  of  training  scenarios  that  encompass  these  behaviors  should  be  developed. 
Although  each  may  address  different  skills,  the  combined  set  should  include  all  the  relevant  skills. 

After  training  scenarios  have  been  identified,  personnel  must  be  instructed  in  team  skills.  Teams  should  be  trained  in  the 
specific  details  of  team-related  tasks  including  how  the  roles  of  individual  members  relate,  how  the  performance  of  the  team 
depends  upon  the  specific  performance  of  the  individuals,  and  what  makes  the  team’s  task  different  from  the  sum  of  the 
individual  ones  (McIntyre  et  al.,  1988,  cited  in  Swezey  and  Llaneras,  1997). 

Salas  and  Cannon-Bowers  (1995,  cited  in  Swezey  and  Llaneras,  1997)  identified  three  generic  methods  for  training  teams: 
information-based,  demonstration-based,  and  practice-based.  The  information-based  methods  involve  the  presenting  of 
facts  and  knowledge  via  lectures,  discussions,  and  similar  techniques.  These  low-cost,  group-based  delivery  methods  may 
be  used  to  help  team  members  understand  what  is  expected  of  them,  what  to  look  for  in  specific  situations,  and  for 
exchanging  general  knowledge. 

The  demonstration-based  methods  show,  rather  than  describe,  the  desired  team  behaviors.  These  behaviors  are  acquired 
through  modeling  (learning  by  example).  For  example,  a  crew  could  be  videotaped  while  responding  to  a  scenario  in  a  way 
that  demonstrated  vital  team  behaviors.  A  training  instructor  could  then  show  the  videotape  to  a  new  crew  pointing  out 
their  effective  behaviors  (Mumaw  et  al.,  1994).  Such  methods  provide  shared  mental  models  among  team  members  (i.e.,  a 
shared  understanding  of  such  things  as  the  current  state  of  the  task,  the  needs  and  expectations  of  crew  members,  and 
required  control  actions).  In  addition,  these  methods  give  examples  of  how  individual  members  are  expected  to  handle 
themselves  during  complex,  dynamic  situations. 

The  practice-based  methods  for  team  training  use  participatory  activities,  such  as  role  playing,  that  provide  opportunities  to 
perform  and  obtain  feedback  on  activities  associated  with  specific  learning  objectives.  Using  repeated  practice  sessions,  the 
trainees  gradually  achieve  the  desired  level  of  performance. 
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5.3.3.6  Evaluation  of  Training  Systems 

Childs  (1996)  identifies  three  evaluation  phases  for  assessing  training  programs:  formative,  summative,  and  operational; 

Formative  Evaluation  -  Formative  evaluation  is  applied  to  the  training  program  as  it  develops  and  may  continue  through  to 
its  delivery.  Formative  evaluation  assesses  the  effectiveness  and  efficiency  of  the  training  system.  Effectiveness  is  related  to 
the  relevancy  of  the  knowledge,  skills,  and  abilities  (KSAs)  that  are  addressed.  Efficiency  is  related  to  the  amount  of 
training  resources  used  to  achieve  the  desired  level  of  proficiency  in  the  trainees.  From  a  safety  perspective,  training 
effectiveness  may  be  of  more  concern  than  efficiency  because  effectiveness  is  concerned  with  the  acquisition  of  required 
KSAs.  However,  reviews  of  efficiency  measures  may  be  useful  in  determining  the  degree  of  attention  that  is  given  to 
specific  topics. 

Training  system  components  may  be  assessed  by  sampling  and  assessing  the  following: 

•  Training  standards 

•  Instructor’s  effectiveness 

•  Training  methods  and  media 

•  Training  measures  and  metrics 

•  Training  scenarios 

•  Instructional  strategies 

•  Coaching  and  mentoring  techniques 

•  Adequacy  of  training  products  (e.g.,  courseware)  and  services  relative  to  training  standards 

•  Potential  for  transfer  of  KSA  to  the  job  setting 

•  Trainee’s  retention  of  knowledge  and  skills. 

Childs  (1996)  suggests  using  pilot  tests  (tryouts)  to  assess  the  validity  of  training  program  courseware.  In  such  tests, 
instructors  use  the  materials  to  train  a  representative  sample  of  surrogate  trainees.  Childs  offers  the  following  evaluation 
criteria  for  examining  the  instructional  materials  and  instruction  delivery: 

•  Courses,  lessons,  and  tests  are  structured  to  facilitate  the  learning  process 

•  Length  of  courses,  lessons,  and  tests  matches  the  objectives  and  level  of  detail  required  for  mastering  KSA 

•  Methods  and  media  are  effective  and  linked  to  objectives 

•  Trainees’  interactions  with  the  courseware  provide  the  necessary  feedback 

•  Sequence  and  pacing  of  delivery  holds  trainees’  interest 
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•  Trainees  are  assimilating  the  intended  KSAs. 

These  criteria  may  be  evaluated  through  questionnaires  and  focus  groups,  or  analyses  of  courseware  test  items. 

Summative  Evaluation  -  A  summative  evaluation  is  conducted  when  training  is  completed  or  when  the  trainee  begins  the 
job  for  which  the  training  was  conducted.  The  evaluation  assesses  the  fit  between  the  training  system  and  the  initial 
requirements  for  operational  performance  (i.e.,  the  degree  to  which  graduates  can  fulfil  all  the  requirements  on  the  job),  as 
well  as  specific  tasks  under  specified  conditions.  Performance  is  evaluated  against  predefined  quantitative  or  qualitative 
standards. 

Operational  Evaluation  -  Operational  evaluation  is  conducted  as  part  of  the  job  setting  to  verify  that  the  trainee  has 
acquired,  and  can  use  the  necessary  KSAs.  It  evaluates  whether  training  satisfies  operational  requirements  considering 
whether  graduates  are  meeting  or  exceeding  their  job  requirements,  specific  training  components  are  facilitating  job 
performance,  job  standards  are  consistent  with  training  objectives,  and  whether  training  could  be  better. 

5.3.4  Summary  of  Interface  Design  Phase 

This  phase  addresses  HFE  considerations  associated  with  designing,  developing,  and  testing  user  interfaces  associated  with 
plant  upgrades.  The  interfaces  include  the  HSI,  procedures,  and  training. 

The  review  of  the  HSI  design  process  should  ensure  that  design  requirements  were  appropriately  translated  into  the  detailed 
HSI  design  by  systematically  applying  HFE  principles  and  criteria.  Attention  should  be  given  to  human  performance 
considerations  related  to  the  user’s  adaptation  to  the  new  design,  including  automaticity,  consistency  between  new  and 
replaced  HSI  components  and  between  the  new  ones  and  the  rest  of  the  HSI,  and  the  functional  integration  of  HSI 
components.  Human  performance  should  be  addressed  in  developing  and  applying  human  factors  guidance  documents, 
tests,  and  evaluations. 

Plant  procedures  are  reviewed  to  ensure  that  any  modifications  reflect  the  characteristics  of  the  upgraded  plant,  and  that 
they  are  properly  integrated  with  the  rest  of  the  procedures.  The  review  should  consider  the  procedures’  content,  format, 
and  integration.  The  extent  of  the  evaluation  should  reflect  the  changes  in  personnel  tasks  and  their  safety  significance. 

Developing  training  requires  systematically  considering  the  skills  and  knowledge  required  of  operators  using  the  upgraded 
systems.  This  process  is  driven  by  a  recognition  of  required  personnel  characteristics,  rather  than  the  physical 
characteristics  of  the  new  design.  Hence,  training  development  is  similar  to  the  top-down  analyses  described  in  the 
requirements’  analysis  phase  (Section  5.2).  Two  key  focuses  for  HFE  reviews  are  to  identify  the  new  knowledge  and  skills 
required  of  plant  personnel,  and  the  appropriate  means  for  conveying  them  to  personnel.  This  section  identified  categories 
of  knowledge  and  skills  associated  with  upgrades  and  discussed  their  acquisition. 

5.4  Veriflcation  and  Validation  Phase 

This  section  covers  the  verification  and  validation  (V&V)  of  NPP  upgrades.  V&V  evaluations  seek  to  comprehensively 
determine  that  an  implemented  design  conforms  to  HFE  design  principles  and  enables  personnel  to  successfully  carry  out 
their  tasks  to  achieve  safety  and  other  operational  goals.  Element  10,  Human  Factors  Verification  and  Validation,  found  in 
Section  1 1  of  NUREG-071 1  identifies  the  following  five  V&V  activities: 

•  HSI  Task  Support  Verification  -  A  check  to  ensure  that  there  are  HSI  components  addressing  all  identified  tasks 
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•  HFE  Design  Verification  -  A  check  to  determine  whether  the  design  of  each  HSI  component  reflects  HFE 
principles,  standards,  and  guidelines 

•  Integrated  System  Validation  -  Performance-based  evaluations  of  the  integrated  design  to  ensure  that  the  HFE/HSl 
supports  safe  operation  of  the  plant 

•  Human  Factors  Issue  Resolution  Verification  -  A  check  to  ensure  that  the  HFE  issues  identified  during  the  design 
were  accepted  and  resolved 

•  Final  Plant  HFE/HSl  Design  Verification  -  A  check  to  ensure  that  the  final  in-plant  installation  of  the  upgrade 
conforms  to  the  design  resulting  from  the  HFE  design  process  and  the  V&V. 

NUREG-071 1  suggests  that  while  activities  generally  should  be  performed  in  the  order  listed,  the  process  is  iterative. 

NUREG-071 1  has  comprehensive  guidance  for  V&V  of  the  entire  HSI.  In  addition,  NUREG-0700  gives  detailed 
methodological  considerations  for  HSI  task  support  verification  and  HFE  design  verification.  For  an  HFE  review  of  an 
upgrade,  all  five  of  these  V&V  activities  may  not  be  necessary.  For  example,  verification  that  a  human  factors  issue  was 
resolved  may  not  be  necessary  if  no  HFE  problems  are  found  in  the  preceding  V&V.  Also,  the  scope  and  level  of  analysis  of 
each  V&V  should  be  consistent  with  the  scope  and  potential  safety  significance  of  the  upgrade.  The  following  describes 
considerations  associated  with  the  first  three  activities,  followed  by  a  discussion  of  V&V  specifically  related  to  installing 
upgrades. 

5.4.1  HSI  Task  Support  Verification 

The  objective  of  this  review  is  to  ensure  that  the  design  provides  all  necessary  alarms,  displays,  and  controls  to  support  all 
identified  personnel  tasks.  It  should  verify  that  all  aspects  of  the  HSI  (e.g.,  controls,  displays,  procedures,  and  data 
processing)  required  to  accomplish  personnel’s  tasks  and  actions,  as  defined  by  the  task  analysis,  emergency  operating 
procedure  analysis,  and  the  risk-important  actions  of  the  PRA  and  HRA,  are  available  through  the  HSI.  In  addition,  it 
should  be  ascertained  that  the  HSI  does  not  include  unnecessary  information,  displays,  controls,  or  other  features. 

Verification  of  HSI  task  support  should  be  conducted  for  all  upgrades  that  are  important  to  safety  and  affect  the  role  of  plant 
personnel  as  described  in  Section  1 1.4.2  of  NUREG-071 1. 

5.4.2  HFE  Design  Verification 

The  objective  of  this  review  is  to  ensure  that  the  design  conforms  to  HFE  principles,  guidelines,  and  standards.  All  HFE 
aspects  of  the  upgrade  HSI  should  be  examined  to  ensure  that  the  design  is  appropriate  for  personnel’s  task  requirements 
and  operational  considerations,  and  as  defined  by  design  specifications.  Deviations  should  be  justified  based  on  documented 
rationales,  such  as  trade  studies,  literature  evaluations,  operational  experience,  and  tests  and  experiments  (see  NUREG- 
071 1).  HFE  design  should  be  verified  for  all  upgrades  of  the  HSI  and  its  components  that  are  significant  to  safety  and  affect 
the  roles  of  plant  personnel;  its  scope  should  also  include  the  interactions  of  upgraded  components  with  the  rest  of  the  HSI. 

When  verifying  HFE  design,  it  is  important  to  understand  personnel  tasks  within  upgraded  plant  so  that  HFE  principles, 
guidelines,  and  standards  may  be  interpreted  and  applied  properly.  As  described  under  fimction-requirements  analysis  and 
function  allocation,  people’s  roles  may  change  after  changes  to  the  HSI  or  plant  systems.  Therefore,  an  important  step  is  to 
characterize  the  changes  in  components  and  personnel  functions  and  tasks  in  the  upgraded  plant  before  applying  HFE 
parameters.  This  characterization  should  reflect  functions  and  tasks  identified  by  evaluations  earlier  in  the  design  process. 


5-39 


NUREG/CR-6637 


5  TECHNICAL  BASIS  DEVELOPMENT:  DESIGN  PROCESS 


The  methodology  for  HFE  design  verification  in  NUREG-0700,  Rev.  1  suggests  ways  for  characterizing  an  HSI.  In 
addition,  the  guidance  and  technical  basis  documents  developed  under  this  and  other  NRC  projects  provide  frameworks  for 
revealing  important  design  characteristics  for  specific  HSI  technologies.  When  reviewing  an  upgrade,  these  frameworks  can 
identify  those  features  of  the  HSI  most  important  to  personnel  performance  which  should  be  addressed  by  the  review.  These 
frameworks  can  also  help  the  reviewer  in  interpreting  HFE  guidance  because  they  link  HSI  design  characteristics  and 
human  performance  and  so  highlight  the  importance  of  particular  characteristics. 

Another  consideration  in  HFE  design  verifications  is  the  need  to  focus  on  the  consistency  of  HSI  upgrades  with  the  rest  of 
the  HSI.  The  high-level  HSI  design  review  principle  of  consistency  in  NUREG-0700,  Rev.  1  (NRC,  1996,  p.  A-2)  states 

There  should  be  a  high  degree  of  consistency  between  the  HSI,  the  procedures,  and  the  training  systems.  At  the  HSI,  the 
way  the  system  functions  and  appears  to  the  operating  crew  always  should  be  consistent,  reflect  a  high  degree  of 
standardization,  and  be  fully  consistent  with  procedures  and  training. 

Inconsistencies  between  the  upgrade  and  other  aspects  of  the  HSI  were  discussed  in  Section  5.3.1,  HSI  Design  and  Testing; 
they  should  also  be  addressed  in  HFE  design  verification  of  the  final  design. 

5.4.3  Integrated  System  Validation 

The  objective  of  this  review  is  to  ensure  that  the  design  can  be  effectively  operated  by  personnel  for  all  requirements.  A  key 
consideration  to  evaluate  user-system  interaction  within  the  context  of  the  integrated  (i.e.,  entire)  HSI  during  realistic 
scenarios;  extensive  guidance  on  validating  integrated  systems  is  provided  in  NUREG-071  land  NUREG-0700,  Rev.  1,  and 
its  technical  basis  is  discussed  in  NUREG/CR-6393  (O’Hara,  Stubler,  Higgins,  and  Brown,  1997).  These  documents 
address  the  comprehensive  review  of  the  HSI,  including  the  entire  main  CR  and  portions  of  the  HSI  that  are  outside  of  the 
CR,  such  as  local  control  stations.  This  guidance  assumes  that  a  new  HSI  design  is  being  evaluated,  but  depending  upon 
the  scope  and  safety  significance  of  an  HSI  upgrade,  such  a  comprehensive  review  may  not  be  necessary.  Much  of  the 
procedural  guidance  describes  ways  to  represent  characteristics  of  future  work  environments,  including  the  installed  design 
and  anticipated  users.  However,  a  review  of  an  upgrade  will  involve  an  existing  facility  with  an  operating  history, 
established  training  programs  and  procedures,  and  experienced  personnel.  This  should  be  kept  in  mind  during  validation 
tests. 

The  following  describes  four  considerations  for  applying  integrated  system  validation  to  HSI  upgrades:  applicability, 
personnel  training,  scenarios,  and  validation  testbed. 

Applicability  -  Integrated  system  validation  should  be  reviewed,  in  accordance  with  Section  1 1 .4.4  of  NUREG-071 1,  for  all 
upgrades  that  are  significant  to  safety  and  may  (1)  change  personnel’s  tasks;  (2)  change  task  demands,  such  as  their 
dynamics,  complexity,  or  workload;  or  (3)  interact  with  or  affect  other  HSI  components  in  ways  that  may  lower 
performance.  Personnel  tasks  should  include  those  identified  as  risk  important  in  HRAs  and  PRAs.  Integrated  system 
validation  may  not  be  needed  when  an  upgrade  results  in  minor  changes  to  tasks  that  may  reasonably  be  expected  to  have 
little  or  no  overall  effect  on  workload  and  the  likelihood  of  error.  The  scope  of  the  integrated  system  validations  should 
encompass  the  upgrade  and  its  interactions  with  the  rest  of  the  plant. 

Personnel  Training  -  NUREG-07 1 1  states  that  plant  personnel  who  participate  in  the  validation  study  should  be  trained  to 
ensure  that  their  knowledge  of  the  operator’s  role,  concept  of  operation,  plant  design,  and  use  of  the  HSI  is  representative  of 
anticipated  users.  This  training  should  address  the  design  characteristics  and  personnel  requirements  associated  with  the 
upgraded  plant  rather  than  with  its  former  configuration. 
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NUREG-07 1 1  also  states  that  plant  personnel  should  be  trained  to  near  asymptotic  performance  (i.e.,  stable,  not 
significantly  changing  from  trial  to  trial)  and  tested  before  the  actual  validation  trials.  Asymptotic  training  assumes  that 
personnel’s  performance  improves  quickly  as  a  function  of  training,  but  then  levels  off  after  a  high  degree  of  skill  is 
attained. 

Section  5.3.1  discussed  the  concept  of  transfer  effects.  When  new  HSI  components  are  not  consistent  with  the  ones  they 
replace,  then  negative  transfer  may  occur.  That  is,  performance  may  be  slowed  and  errors  may  occur  when  people  perform 
tasks  on  the  new  HSI  components  using  actions  and  strategies  that  are  appropriate  for  the  old  design.  Personnel  should 
have  sufficient  training  with  the  new  design  to  extinguish  old  response  patterns  and  acquire  new  ones.  In  addition,  the 
principles  of  good  HSI  design  dictate  that  potential  errors  are  anticipated  and  addressed  by  either  designing  upgrades  that 
are  consistent  with  the  old  components  or  by  providing  features  to  protect  against  errors.  The  validation  study  should 
examine  the  combined  effects  of  HSI  design  and  personnel  training;  this  can  only  be  done  if  personnel  are  adequately 
trained. 

Test  Objectives  and  Scenarios  -  NUREG-07 1 1  states  that  detailed  objectives  should  be  developed  for  validation  tests.  They 
should  address  the  roles  of  plant  personnel,  shift  staffing,  adequacy  of  HSI  support  for  people’s  allocated  functions,  their 
ability  to  perform  tasks  within  time  and  performance  criteria,  adequacy  of  HSI  features  for  their  intended  functions,  the 
ability  of  personnel  to  make  effective  transitions  between  features,  and  the  tolerance  of  the  integrated  to  HSI  failures  of 
individual  features.  The  objectives  should  also  identify  factors  that  may  negatively  affect  integrated  system  performance. 
These  objectives  should  be  reflected  in  scenarios  developed  to  validate  the  upgrades.  In  addition,  the  scenarios  should  focus 
on  dimensions  of  personnel  performance  that  are  of  particular  concern  in  designing  upgrades  and  were  addressed  in  the 
analysis  and  interface  design  phases.  These  dimensions  include  the  following: 

•  New  and  changed  personnel  functions  resulting  from  the  upgrade 

•  Use  of  upgrades  in  coordination  with  the  rest  of  the  HSI 

•  Crew  coordination. 

These  scenarios  should  exercise  the  HSI  features  developed  to  address  the  specific  design  considerations  identified  in  the 
analysis  and  interface-design  phases.  The  scenarios  should  provide  opportunities  to  observe  personnel  performance  under 
high-demand  conditions  to  ensure  these  HSI  characteristics  are  compatible  with  cognitive  skills  and  capabilities. 

Validation  Tes/bci/— NUREG-07 1 1  and  NUREG/CR-6393  (O’Hara,  Stubler,  Higgins,  and  Brown,  1997)  give  extensive 
guidance  on  the  characteristics  of  the  testbed  to  ensure  that  the  HSI  and  work  environment  are  represented  with  an 
appropriate  level  of  fidelity.  NUREG-07 1 1  assumes  a  high-fidelity  simulator  will  be  used  to  represent  the  main  CR.  Seven 
dimensions  of  fidelity  for  the  main  CR  testbed  are  described:  HSI  completeness,  HSI  physical  fidelity,  HSI  functional 
fidelity,  data  completeness  fidelity,  data  content  fidelity,  data  dynamics  fidelity,  and  environment  fidelity. 

Existing  NPPs  are  required  to  have  full-scope,  high-fidelity  simulators  for  training  personnel.  They  can  provide  an 
excellent  environment  for  validation  studies.  However,  HSI  upgrades  are  generally  not  installed  in  these  simulators  until 
after  the  main  CR  has  been  upgraded.  Licensees  are  generally  required  to  change  the  training  simulators  within  a  specified 
’penod  following  the  installation  of  upgrades.  However,  they  may  be  required  to  upgrade  the  simulators  prior  to  upgrading 
the  main  CR  as  a  condition  for  licensing  an  upgrade  if  it  involves  changing  technical  specifications  or  specifications  in  the 
plant’s  safety  analysis  report  (SAR)  or  NRC’s  safety  evaluation  report  (SER).  Therefore,  depending  upon  the  potential 
safety  significance  and  other  licensing  considerations,  validation  of  an  upgrade  may  be  conducted  in  the  training  simulator. 
If  the  behavior  of  systems  is  expected  to  change  after  the  upgrade,  then  the  plant  simulation  model  should  be  modified 
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accordingly.  This  simulation  may  be  based  on  engineering  estimates  of  plant  behavior  if  actual  performance  data  do  not 
exist.  NUREG-071 1  gives  criteria  for  the  fidelity  of  the  plant  simulation  (i.e.,  data  dynamics  fidelity)  to  ensure  that  the 
plant  behavior  is  accurately  portrayed. 

If  trials  are  not  performed  in  the  training  simulator,  then  other  suitable  test  beds  must  be  used.  The  main  control  room  may 
be  acceptable,  although  it  may  only  be  suitable  for  validation  trials  that  require  limited  levels  of  fidelity.  For  example,  if 
walk-through  exercises  are  conducted  in  the  main  control  room,  it  may  be  difficult  to  represent  factors  that  are  important  to 
performance,  such  as  time  stress,  changes  in  conditions,  and  requirements  for  the  coordinated  use  of  multiple  HSI 
components.  While  such  evaluations  may  be  helpful  for  assessing  considerations  related  to  HSI  task  support  verification 
and  HFE  design  verification,  they  are  of  limited  value  for  integrated  system  validation  for  an  HSI  upgrade. 

Another  approach  for  validating  upgrades  is  to  use  part-task  simulators,  devices  that  simulate  the  behavior  of  user  interfaces 
and  plant  systems  for  a  limited  portion  of  the  HSI.  Part-task  simulators  may  be  appropriate  for  validating  limited  upgrades. 
For  example,  they  may  be  appropriate  when  the  part-task  simulator  can  represent  all  interactions  with  HSI  components  and 
plant  systems  related  to  the  upgrade. 

NUREG-07 1 1  indicates  that  a  simulation  or  mockup  should  be  considered  for  assessing  timely  and  precise  human  actions 
undertaken  at  complex  HSI  components  that  are  remote  from  the  main  CR.  The  important  characteristics  of  task-related 
HSI  components  and  the  task  environment  should  be  considered  (e.g.,  lighting,  noise,  heating  and  ventilation,  and 
protective  clothing  and  equipment).  Two  types  of  complex  HSI  components  suitable  for  this  type  of  evaluation  are  local 
control  centers  and  maintenance  interfaces  for  digital  systems.  Brown,  Higgins,  and  O’Hara  (1994)  discussed  HFE 
considerations  associated  with  local  control  stations.  A  review  on  the  maintenance  of  digital  systems  found  that  human 
performance  is  a  leading  cause  of  failures  of  these  systems  (Stubler,  Higgins,  and  Kramer,  2000).  Design  features 
associated  with  errors  in  maintenance  for  digital  systems  include  soft  controls  in  the  user  interface,  multiple  modes  for 
operation  and  testing,  and  automatic  switching  between  redundant  processors.  In  addition,  the  trend  toward  performing 
maintenance  while  the  plant  is  at  power  can  increase  the  likelihood  of  maintenance  errors  and  their  potential  consequences 
(e.g.,  initiation  of  plant  safety  systems  and  plant  trips). 

When  developing  validation  testbeds  for  operational  and  maintenance  tasks  at  local  control  stations  and  local  maintenance 
interfaces,  the  following  should  be  considered; 

•  Representation  of  the  control  and  display  characteristics,  including  dynamic  plant  information,  the  operation  of 
controls,  and  the  behavior  of  automatic  HSI  features  (e.g.,  automatic  mode  changes). 

•  Representation  of  system  faults 

•  Representation  of  HSI  faults 

•  Representation  of  communications  with  CR  personnel 

•  Representation  of  task  environmental  factors  that  may  affect  performance. 
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5.4.4  Verification  and  Validation  Considerations  in  Installing  Upgrades 

HFE  verification  and  validation  may  be  affected  by  the  way  in  which  an  upgrade  is  implemented.  In  reviewing  Canadian 
NPP  event  reports,  Ragheb  (1992)  found  about  50  events  that  were  caused  by,  or  that  occurred  during,  the  design  and 
implementation  of  plant  modifications.  The  causes  included  the  following: 

•  Faulty  change  in  design 

•  Use  of  faulty  temporary  solutions  during  delays  to  the  final  upgrade 

•  Failure  to  verify  or  test  changes  after  implementation  or  installation 

•  Leaving  equipment  in  an  unsafe  state  when  making  a  change 

•  Improper  control  of  changes  in  design 

•  Improper,  inadequate,  or  outdated  documentation  supporting  the  design  change,  such  as  design  manuals,  operating 
manuals,  and  work  plans. 

The  review  underscored  the  importance  of  tests  and  evaluations  when  developing  and  implementing  upgrades. 

As  described  in  Section  5.1.2,  upgrades  are  not  always  implemented  all  at  once.  Instead,  there  may  be  an  interim  period  in 
which  the  plant  differs  from  both  its  original  configuration  and  desired  final  one.  Section  5.1.2  describes  four  different 
patterns  for  implementing  upgrades:  complete  replacement,  phased  implementation,  dual  installation  with  delayed  switch¬ 
over,  and  dual  installation  with  no  switch-over.  New  opportunities  for  personnel  error  may  occur  during  the  transition  from 
the  old  HSI  to  the  new  one.  Therefore,  when  verifying  and  validating  upgrades,  the  following  factors  should  be  considered. 

•  HFE  deficiencies  in  temporary  HSI  and  plant  configurations  -  Temporary  configurations  of  HSI  or  system 
components  may  be  created  during  phased  implementations  or  the  dual  installation  of  upgrades.  That  is,  the  plant 
may  be  operated  using  combinations  of  HSI  components  and  system  components  that  differ  from  both  the  original 
design  and  the  final  design.  Because  they  are  temporary,  these  configurations  may  not  have  the  same  level  of  HFE 
review  as  did  the  original  or  final  configurations.  Therefore,  the  design  may  have  negative  effects  on  performance. 
For  example,  the  presence  of  old  and  new  components  in  the  HSI  may  increase  its  overall  complexity,  and  so 
impose  special  demands  on  personnel  for  retrieving  information  and  executing  control  actions.  Also,  if  the 
temporary  configuration  of  HSI  systems  is  not  fully  compatible  with  that  of  plant  systems,  then  the  state  of  the 
plant’s  equipment  may  not  be  accurately  conveyed  to  personnel. 

•  Inadequate  temporary  fixes  -  When  the  HSI  or  plant  systems  are  modified,  it  is  sometimes  necessary  to  make 
temporary  “fixes”  to  address  problems  (“bugs”)  peculiar  to  the  current  configuration.  As  Ragheb  describes,  these 
“fixes”  may  pose  new  challenges  to  personnel  performance. 

•  Continuous  change  -  The  phased  implementation  of  upgrades  may  create  a  period  when  personnel  must 
continually  adapt  to  the  changing  characteristics  of  the  plant  and  HSI,  imposing  high  demands  on  them  for 
keeping  their  knowledge  of  the  plant  up-to-date.  Performance  may  be  degraded  when  users  apply  strategies 
incompatible  with  the  current  design. 
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•  Improperly  planned  or  implemented  changes  -  As  Ragheb  describes,  plant  events  may  occur  as  a  result  of  upgrades 
that  are  inadequately  planned  or  poorly  implemented. 

•  Inadequate  procedures  and  technical  documentation  -  Ragheb  also  states  that  plant  procedures  and  other 
supporting  documentation  may  not  accurately  reflect  the  current  configuration  of  an  upgrade.  When  the  systems 
and  HSI  are  continually  changing,  as  a  phased  implementation,  this  consideration  may  be  particularly  important. 

Attention  should  be  given  to  evaluating  these  HFE  features  of  temporary  configurations  of  the  HSI  and  plant  systems.  The 
following  are  considerations  related  to  HSI  task  support  verification,  HFE  design  verification,  and  integrated  system 
validation: 

HSI  Task  Support  Verification  -  HSI  task  support  verification  should  address  temporary  configurations  and  deactivated  HSI 
components  left  in  the  HSI.  Temporary  configurations  should  undergo  HSI  task  support  verification  if  they  will  be  used  by 
operations  or  maintenance  personnel  when  the  plant  is  not  shut  down.  In  applying  the  NUREG-071 1  criteria  for  this 
verification,  attention  should  be  given  to  unique  information  and  control  requirements  that  may  result  from  temporary 
configurations.  It  should  ensure  that  if  the  temporary  configurations  of  plant  systems  have  any  special  characteristics  (e.g., 
unique  response  rates,  control  modes,  or  operating  ranges)  that  impose  special  monitoring  or  control  requirements,  then  the 
HSI  should  have  the  information  and  control  capabilities  to  satisfy  them.  Also,  this  verification  should  examine  the 
characteristics  of  HSI  components  that  may  be  installed  temporarily  to  ensure  that  they  also  provide  all  needed  control  and 
display  capabilities. 

Criterion  2  of  Section  1 1 .4.2  of  NUREG-071 1  states  that  the  HSI  should  not  contain  any  information,  displays,  or  controls 
that  do  not  support  operator  tasks.  Therefore,  this  verification  should  identify  deactivated  HSI  components  that  may  have 
potentially  negative  effects  on  performance.  Examples  include  obstructing  the  view  of  important  information,  or  adding 
visual  clutter  that  may  interfere  with  monitoring  tasks.  If  further  evaluations  are  needed  to  assess  the  severity  of  effects  on 
performance,  then  these  deactivated  HSI  components  should  undergo  HFE  design  verification  or  integrated  system 
validation. 

HFE  Design  Verification  —  HFE  guidance  should  be  applied  to  the  review  of  temporary  configurations,  especially  to  the 
consistency  of  HSI  upgrades  with  the  rest  of  the  HSI.  Also,  the  consistency  of  “fixes”  with  HFE  principles,  standards,  and 
guidelines  should  be  examined. 

Integrated  System  Validation  -  Validation  tests  should  examine  the  potential  effects  on  personnel  performance  stemming 
from  the  presence  of  both  the  old  and  new  components  in  the  HSI.  For  example,  if  both  are  functional,  as  in  the  case  of 
dual  installation  with  no  switch-over,  problems  may  occur  due  to  differences  in  the  methods  of  operation.  Section  5.3.1 
described  the  potential  problems  associated  with  alternating  use  of  inconsistent  HSI  designs,  this  may  be  particularly 
important  when  the  same  information  or  control  capabilities  can  be  accessed  from  different  devices  with  different  means  of 
operation.  Personnel  may  have  difficulty  recalling  the  correct  use  of  one  device  if  the  other  was  used  recently.  Also,  the 
strategies  that  personnel  employ  when  using  a  device  (e.g.,  ways  of  rapidly  scanning  the  displayed  information  or  executing 
control  actions)  during  specific  operations  may  not  translate  well  between  different  devices  and,  therefore,  may  result  in 
errors. 

Validation  tests  should  evaluate  temporary  configurations;  considerations  include  the  adequacy  of  information  and  control 
capabilities,  the  integration  of  controls  and  displays,  and  the  adequacy  of  training,  procedures,  and  technical  documentation. 
Particular  attention  should  be  given  to  the  consistency  of  these  HSI  upgrades  with  the  rest  of  the  HSI,  and  to  the  conformity 
of  temporary  “fixes”  to  HFE  principles,  standards,  and  guidelines. 
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Safety  significance  is  an  important  consideration  in  determining  those  conditions  that  should  be  evaluated  through 
integrated  system  validation.  Two  important  factors  are  the  risk  importance  of  personnel’s  actions,  and  the  probability  that 
errors  may  occur.  For  temporary  configurations,  the  error  probability  is  affected  by  the  time  that  a  particular  configuration 
may  exist.  As  personnel’s  exposure  to  the  temporary  configuration  is  reduced,  the  likelihood  of  an  error  is  also  reduced. 
Therefore,  validation  tests  may  not  be  required  for  some  temporary  configurations  if  risk-based  analyses  show  that  they  are 
not  risk  important.  The  basis  of  the  analyses  may  include  the  length  of  time  that  personnel  will  be  exposed  to  these 
configurations. 

Summary  of  Evaluation  Phase 

The  installation  and  evaluation  phase  addresses  the  HFE  verification  and  validation  of  the  final  upgrade’s  design,  as 
described  in  NUREG-071 1.  However,  the  V&V  methodology  and  criteria  must  be  focused  on  the  relevant  characteristics  of 
upgrades.  Particular  attention  should  be  given  to  understanding  how  personnel  tasks  will  be  changed  by  the  upgrades  so 
that  HFE  guidance  may  be  properly  applied,  to  assessing  the  consistency  of  the  upgrades  with  the  rest  of  the  HSI,  and  to  the 
suitability  of  any  temporary  configurations  during  incremental  upgrades. 

5.5  Organizational  Considerations  for  Designing  and  Implementing  HSI  Upgrades 

When  plants  are  upgraded  or  modified,  personnel  and  organizations  must  adapt  to  the  change.  The  failure  of  personnel  to 
accept  changes  in  technology  in  their  work  environment  was  identified  as  a  problem  in  many  domains  (Medsker  and 
Campion,  1997;  Price,  1990).  In  some  NPPs,  personnel  had  difficulty  adapting  to  the  new  HSI  technologies  when  they 
were  introduced.  The  following  are  two  examples: 

•  A  utility  introduced  a  new  NPP  that  featured  a  CR  that  was  quite  different  from  its  predecessor  designs.  The  plant 
had  digital  control  systems  and  an  HSI  which  prominently  featured  a  CRT-based  display  system  and  computer- 
based  input  devices  (e.g.,  light  pens  and  keyboards).  Many  operators  came  from  an  older  NPP  that  was  being 
permanently  shut  down.  Even  though  they  were  involved  in  developing  the  CR  and  many  HFE  evaluations  were 
held,  acceptance  was  not  universal,  especially  in  using  the  HSI  (Fenton  and  Duckitt,  1991). 

•  A  NPP,  visited  as  part  of  this  research  project,  set  up  a  computer-based  system  for  analyzing,  storing,  and 
retrieving  chemistry  data.  About  six  months  later,  management  stated  that  although  the  system  provided  many 
benefits,  about  one-half  of  the  users  had  problems  with,  or  expressed  some  resistance  to,  the  new  system.  They  had 
difficulty  with  the  change  in  computer  technology  because  the  new  digital  system  had  characteristics  different  from 
the  old  computer  system  formerly  used  for  chemical  analysis.  Difficulties  in  adapting  apparently  were  not  related 
to  job  level,  but  were  evenly  distributed  among  engineers,  technicians,  and  other  support  personnel  (O’Hara, 

Stubler,  and  Higgins,  1996). 

Resistance  to  technology  changes  also  has  been  encountered  in  fossil-fuel  power  plants,  chemical  plants,  and 
manufacturing: 

•  When  a  multi-unit  fossil  power  plant  upgraded  some  of  its  control  centers,  some  operators  were  unable  to  adapt  to 
the  CRT-based  control  centers  that  replaced  the  original  conventional  ones.  Consequently,  some  operators  were 
permanently  assigned  to  conventional  control  centers,  while  others  were  rotated  between  the  computer-based  and 
conventional  control  centers  (Barnes  and  Ryan,  1995). 
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•  At  one  multi-unit  chemical  facility,  visited  during  this  research  project,  operators  requested  transfers  to  an  older 
unit  that  had  an  analog  control  system,  rather  than  remain  at  the  units  that  were  to  be  upgraded  with  digital  control 
systems  (O’Hara,  Stubler,  and  Higgins,  1996). 

•  Discussions  with  personnel  from  a  foreign  power-generation  organization,  also  part  of  this  research  project, 
indicated  that  significant  resistance  from  operators  was  encountered  at  one  of  its  fossil  plants  when  digital  I&C 
systems  were  installed  and  the  CR  was  converted  to  one  featuring  computer-based  operator  consoles  (O’Hara, 
Stubler,  and  Higgins,  1996). 

•  An  estimated  50%  to  75%  of  new  manufacturing  technologies  in  the  United  States  have  failed.  The  disregard  for 
human  and  organizational  concerns  was  considered  a  greater  contributor  to  these  failures  than  were  technical 
problems  (Medsker  and  Campion,  1997). 

Resistance  and  nonacceptance  of  changes  in  the  work  environment  are  of  special  concern  in  NPPs  because  of  the 
importance  of  personnel  performance  to  plant  safety.  It  may  result  in  the  failure  of  personnel  to  manage  or  use  new 
technology  effectively.  For  example,  because  of  a  lack  of  acceptance  of  new  automated  systems,  personnel  may  not  actively 
monitor  the  systems  to  ensure  that  the  automation  is  operating  correctly.  Alternatively,  lacking  trust  in  automation, 
personnel  may  attempt  to  take  the  system  out  of  the  automatic  mode  and  control  it  manually.  Furthermore,  because 
personnel  do  not  accept  advanced  HSI  capabilities,  they  may  be  reluctant  to  undertake  interface  management  to  search  for 
additional  information  during  transients,  but  may  instead  depend  upon  alarms  (O’Hara,  Stubler,  and  Nasta,  1997).  Plant 
performance  and  safety  then  may  be  affected. 

From  reviewing  relevant  research,  Medsker  and  Campion  offer  a  set  of  principles  for  reducing  resistance  to  change  by 
involving  personnel  in  the  development  of  the  upgrade.  The  following  were  derived  from  their  principles: 

•  Workers’  involvement  in  the  change  -  Workers  should  be  informed  of  changes  in  advance,  involved  in  diagnosing 
current  problems,  and  in  developing  solutions.  Resistance  may  be  decreased  if  participants  feel  a  sense  of 
ownership  in  the  project  (i.e,  they  do  not  feel  it  is  being  imposed  upon  them). 

•  Strong  support  by  top  management  for  the  change  -  Workers  may  be  more  likely  to  take  the  project  seriously  if 
they  feel  management  is  strongly  committed  to  its  success. 

•  Change  made  consistent  with  workers’  needs  and  existing  values  -  Workers  need  to  see  the  change  as  benefitting 
them.  Resistance  may  be  reduced  if  change  is  perceived  as  something  that  will  reduce  burdens,  offer  interesting 
experiences,  not  threaten  their  autonomy  or  security,  and  not  conflict  with  other  goals  and  values  of  the 
organization.  Resistance  also  may  lessen  if  proponents  of  the  change  empathize  with  those  opposed  to  it  by 
recognizing  valid  objections  and  relieving  unnecessary  fears. 

•  An  environment  of  open,  supportive  communication  -  Resistance  may  fall  if  participants  trust  each  other  and  are 
supported.  Misunderstandings  and  conflicts  should  be  expected  as  a  natural  part  of  the  innovation  process. 
Provisions  should  be  made  for  clarifying  misunderstandings. 

•  Allowance  for  flexibility  -  Resistance  may  be  reduced  if  the  project  remains  open  to  revision  and  reconsideration  as 
experience  is  gained. 

Price  (1990)  states  that  technology  alone  is  unlikely  to  improve  the  performance  of  an  organization  unless  attention  is  given 
to  the  people  who  use  it.  Change  is  likely  to  be  rejected  if  it  is  poorly  understood  by  users,  perceived  as  being  imposed  by 
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management  or  as  being  risky  or  threatening,  or  if  it  does  not  improve  the  work  environment.  The  introduction  of 
automation  can  pose  the  following  particular  problems  in  acceptance: 

•  Workers’  fear  of  automation  that  has  intellectual  functions,  such  as  expert  judgement  and  decision  making 

•  Workers’  fear  of  the  unreliability  of  a  device 

•  Workers  being  held  accountable  for  the  output  of  an  automated  system  when  they  feel  they  have  limited  control 
over  it. 

Price  gives  a  set  of  principles  related  to  the  acceptance  of  automation  by  workers,  based  on  an  earlier  study  of  automated 
commercial  all-weather  landing  systems.  These  principles  are  shown  in  Table  5.4.  They  are  consistent  with  those  offered 
by  Medsker  and  Campion. 

Experiences  in  NPPs  indicate  the  importance  of  making  the  change  consistent  with  the  operator’s  needs.  However, 
experience  also  shows  that  obtaining  comments  from  users  about  their  needs  can  be  difficult  during  the  early  stages  of  a 
design  project.  For  example,  a  NPP  that  was  developing  a  new  computer-based  display  system  wanted  early  feedback  from 
operators  about  the  system  (Roth  and  O’Hara,  1998).  They  installed  an  early  version  of  the  new  display  system  in  the  CR  so 
that  operators  could  become  familiar  with  its  capabilities  and  see  its  information  within  the  context  of  plant  operations. 
Because  this  was  an  early  version,  not  all  data  were  presented,  and,  due  to  some  known  but  uncorrected  software  “bugs,” 
some  messages  it  generated  were  incorrect.  Consequently,  operators  had  difficulty  overlooking  the  limitations  and 
envisioning  the  potential  benefits  of  the  fully  implemented  system.  Instead,  they  expressed  concern  over  the  usefulness  and 
reliability  of  the  system,  based  on  the  limitations.  In  other  words,  the  system  was  not  viewed  as  being  consistent  with  their 
needs  and  existing  values. 

Several  conclusions  can  be  drawn  from  this  experience.  First,  it  may  be  undesirable  to  introduce  a  display  system  or  other 
personnel  aids  into  a  work  situation  until  the  design  deficiencies  have  been  corrected.  Instead,  it  may  be  better  to  introduce 
personnel  to  the  system  in  a  different  setting,  such  as  a  testing  facility.  A  goal  in  designing  the  user  interface  is  to  reduce 
the  workload  associated  with  its  use  (i.e.,  it  is  often  stated  that  the  interface  should  be  “transparent”  to  the  user).  When  the 
device’s  capabilities  are  inoperable  or  when  information  displayed  is  incomplete  or  incorrect,  the  workload  is  likely  to 
increase.  In  addition,  when  the  potential  consequences  of  errors  are  serious,  such  as  in  NPPs,  users  may  react  negatively  to 
incomplete  or  incorrect  information.  Second,  this  experience  also  shows  the  importance  of  telling  people  about  the 
limitations  of  early  designs.  While  early  involvement  by  users  is  generally  considered  desirable,  exposure  to  early  designs 
may  also  lead  to  a  lack  of  acceptance. 

A  model  for  the  successful  transfer  and  application  of  new  technology  in  the  workplace  is  offered  by  Mackie  and  Wyle 
(1988).  The  model  comprises  three  major  activities  which  are  conducted  in  parallel:  communicating  with  potential  users, 
involving  users  in  the  development  process,  and  designing  for  acceptance.  Each  is  described  below. 

•  Communicating  with  potential  users  -  This  activity  begins  early  in  the  design  process.  The  intent  is  to  gather 
information  from  the  intended  users  about  their  needs  and  attitudes  regarding  new  technology  and  keep  users 
informed  about  how  the  planned  technology  will  affect  them.  Information  gathered  from  potential  users  may 
include  operational  needs,  their  understanding  of  new  technologies  including  positive  and  negative  beliefs,  and 
features  considered  desirable  by  the  potential  users.  Information  disseminated  to  the  intended  users  may  include 
operational  needs  currently  being  addressed  or  considered,  new  features  being  developed  or  considered,  and 
potential  benefits  to  the  users. 
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Table  5.4  Principles  for  Users’  Acceptance  of  Automation  for  Landing  Systems 

(Adapted  from  Price,  1990) 

Role  Expectancy 

People  are  generally  accepting  of  system  roles  that: 

•  Provide  opportunities  to  exercise  and  maintain  skills  they  feel  are  important  to  their  maintaining  their 
position  in  their  occupational  and  social  status  system 

•  Allow  initiative  and  flexibility  in  their  practices  and  manner  of  accomplishing  tasks  (i.e.,  work  practices  are 
not  "mechanical") 

•  Provide  opportunities  for  and  allow  learning 
Automation  Acceptance 

People  are  generally  more  accepting  of  automation: 

•  With  increased  exposure  and  experience  with  the  system 

•  If  their  position  has  status,  responsibility,  and  authority 

•  Where  failure  of  the  automated  function  is  not  associated  with  high  consequences  (e.g.,  does  not  endanger 
human  life) 

•  For  tasks  that  must  be  performed  over  long  periods  of  time 

•  For  functions  that  do  not  have  a  large  decision-making  component 


Involving  users  in  the  development  process  -  This  activity  includes  users’  participation  in  the  identification  of 
design  inputs  and  the  development  of  testbeds.  Because  it  requires  the  participation  of  individuals  having  particular 
knowledge  relevant  to  the  new  product  or  system,  this  activity  may  involve  fewer  potential  users  than  the  previous 
activity  -  communicating  with  potential  users.  Two  important  considerations  from  a  program  management 
perspective  are  the  identification  of  key  personnel  from  the  potential  user  population  and  then  their  utilization. 

Key  personnel  may  participate  in  the  development  of  design  inputs,  the  review  of  designs  and  design  concepts,  and 
the  development  of  tests  for  systems  or  subsystems.  In  addition,  some  key  personnel  may  act  as  change  agents  by 
providing  feedback  to  the  general  user  population  regarding  the  developing  design.  For  the  development  of 
testbeds,  potential  users  may  be  involved  in  the  selection  of  the  test  setting  (e.g.,  mockup,  simulator,  or  operational 
setting),  the  development  of  test  scenarios,  and  the  identification  of  criteria  for  selecting  test  participants. 

Designing  for  acceptance  -  This  activity  relates  to  the  engineering  design  of  the  new  product  or  system.  Design 
considerations  identified  through  interactions  with  potential  users  (as  described  in  the  two  previous  activities)  are 
translated  into  design  requirements.  Both  general  criteria  for  user  acceptance  and  specific  criteria  related  to  the 
particular  type  of  innovation  are  addressed.  General  design  criteria  include  consideration  of  operational 
constraints,  operability,  reliability,  compatibility,  maintainability,  supportability,  and  training  concerns.  Specific 


NUREG/CR-6637 


5-48 


5  TECHNICAL  BASIS  DEVELOPMENT:  DESIGN  PROCESS 


criteria  address  the  characteristics  of  particular  design  features,  such  as  those  for  selecting  and  manipulating 
information.  The  activity  addresses  the  suitability  of  these  design  features  for  the  tasks  and  characteristics  of  the 
users. 

The  three  activities  of  the  Mackie  and  Wyle  model  converge  during  a  demonstration  phase,  which  corresponds  to  the  later 
stages  of  Element  7,  Interface  Design,  of  NUREG-071 1.  During  this  phase,  testbeds  incorporating  features  that  were 
developed  during  the  earlier  activities  of  the  process  are  tested  with  the  participation  of  potential  users.  These  tests  are 
intended  to  address  such  considerations  as  operational  validity,  operational  limitations,  reliability,  and  supportability. 

This  process  appears  to  address  many  of  the  points  raised  earlier  by  Price  (1990)  and  Medsker  and  Campion  (1997).  For 
example,  involvement  of  workers  in  the  development  process  can  help  focus  the  new  technology  on  their  needs  and  existing 
values.  The  design  for  acceptance  activity  can  ensure  that  the  design  is  consistent  with  other  organizational  concerns  such 
as  reliability,  maintainability,  and  supportability.  The  model  does  not  explicitly  deal  with  the  need  for  the  support  of  top 
management  for  the  change.  However,  it  is  implied  by  the  degree  of  the  workers’  participation  in  the  process.  Also,  the 
need  for  open,  supportive  communication  and  for  flexibility  are  not  explicitly  covered  but  are  implied  by  the  model; 
however,  probably  both  would  be  necessary  when  integrating  information  from  the  three  activities. 

When  reviewing  an  HSI  upgrade,  some  of  the  considerations  discussed  here  may  be  easier  to  assess  than  others.  For 
example,  the  workers’  involvement  in  planning  the  change  may  be  assessed  by  reviewing  the  design  process  and  noting 
when  operators  and  maintenance  personnel  provided  input;  this  may  include  inputs  at  such  stages  as  the  operating 
experience  review,  requirements  analysis,  tests,  and  evaluations.  The  flexibility  allowed  in  the  development  process  may  be 
assessed  by  noting  the  degree  to  which  the  HSI  design,  procedures,  and  training  were  modified  by  user’s  inputs  or  operating 
experiences.  Top  management’s  support,  communication,  and  the  consistency  of  the  change  with  the  users’  needs  and 
existing  values  may  be  more  difficult  to  assess. 

The  concept  of  participatory  ergonomics  has  received  growing  interest  as  a  means  to  enhance  the  design  of  complex  human- 
machine  systems  (Wilson  and  Haines,  1997;  Noro,  1991;  Sen,  1987).  It  is  defined  as  “...the  involvement  of  people  in 
planning  and  controlling  a  significant  amount  of  their  own  work  activities,  with  sufficient  knowledge  and  power  to 
influence  both  processes  and  outcomes  in  order  to  achieve  desirable  goals”  (Wilson  and  Haines,  1997).  The  potential 
benefits  of  participatory  ergonomics  stem  from  the  user’s  participation  in  design  and  implementation.  Using  participation 
techniques  in  change  analysis,  development,  and  implementation  may  generate  in  the  users  greater  feelings  of  ownership, 
and  subsequently  a  greater  commitment  to  such  changes.  Also,  involving  current  users  in  redesigns  may  improve  the  flow 
of  information  and  the  generation  of  ideas  during  the  process  because  users  may  be  intimately  familiar  with  the  positive  and 
negative  characteristics  of  the  current  situation. 


Despite  the  potential  benefits,  Wilson  and  Haines  caution  that  participatory  techniques  have  limitations  and  potential 
disadvantages.  Some  limitations  are  related  to  the  intentions  and  motivation  of  participants  and  facilitators,  and  others 
have  to  do  with  the  types  of  outpute  that  can  be  produced  by  participation.  Some  situations  require  professional  ergonomic 
expertise  and  cannot  be  adequately  addressed  by  substituting  input  from  users.  Finally,  Wilson  and  Haines  note  that  few 
studies  have  given  solid  evidence  of  the  cost  effectiveness  of  participatory  techniques,  while  anecdotal  evidence  exists  of 
participatory  projects  that  have  fallen  short  of  expectations. 

Thus,  a  better  understanding  is  needed  of  the  causes  of  poor  operator  acceptance,  its  impact  on  plant  operation  and  safety, 
appropriate  countermeasures,  and  appropriate  methodologies  for  dealing  with  these  challenges.  These  considerations  are' 
important  both  for  the  transition  from  traditional  to  computer-based  technologies  and  for  subsequent  upgrades.  The 
flexibility  of  computer-based  systems  and  the  rapid  pace  at  which  they  become  technologically  obsolete  may  increase  the 
rate  at  which  HSIs  are  upgraded  in  future.  New  HSI  design  and  evaluation  processes  may  be  needed  to  ensure  user 
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acceptance,  and  training  processes  may  have  to  become  more  adaptable  to  the  complex,  frequently  changing  computer- 
based  technologies  and  the  diversity  of  HSI  technologies  in  hybrid  plants.  Design  and  implementation  of  HSI  upgrades  will 
need  to  cover  the  broader  range  of  skills  needed  to  operate  hybrid  interfaces,  the  potential  for  negative  transfer  of  skills 
associated  with  upgrades,  and  the  resistance  of  personnel  to  the  associated  new  demands. 

5.6  Unresolved  Issues  For  Further  Research 

This  section  summeu'izes  the  human  performance  issues  associated  with  designing  and  implementing  NPP  upgrades  that 
were  detailed  in  this  review.  These  issues  represent  topics  on  which  research  is  necessary  before  additional  guidance  can  be 
developed. 

The  Effects  of  HSI  Inconsistency  Upon  Alternating  Use  of  HSI  Components 

Empirical  studies  have  found  that  human  performance  may  show  a  greater  decrement  when  different  devices  in  an  HSI  have 
overlapping  similarities  in  their  methods  of  operation  than  when  these  methods  are  very  different  (Tanaka  et  al.,  1991). 
Overlaps  in  operating  computer-based  user  interfaces  can  disrupt  human  performance  by  increasing  the  likelihood  of  errors 
or  increasing  the  amount  of  time  and  mental  effort  needed  to  generate  the  correct  response.  Much  of  the  existing  HFE 
guidance  on  consistency  is  directed  toward  the  appearance  of  interfaces,  and  a  greater  understanding  is  needed  of  the 
dimensions  of  consistency  as  related  to  the  operation  of  computer-based  user  interfaces.  New  methods  for  assessing 
consistency  are  being  developed,  such  as  the  text-editing  model  (TEM)  (Tanaka  et  al.,  1991),  for  predicting  the  effects  of 
different  kinds  of  computer  displays  and  of  interaction-consistent  mappings  on  the  performance  of  computer-based  tasks. 
These  analytical  methods  and  supporting  empirical  studies  require  further  review  to  (1)  identify  dimensions  of  inconsistency 
relevant  to  tasks  performed  by  operations  and  maintenance  personnel  in  NPPs,  and  (2)  develop  techniques  appropriate  for 
reviewing  NPP  HSIs. 

The  Effects  of  HSI  Design  on  Crew  Coordination  and  Cooperation 

Computer-based  HSI  technologies  can  affect  crew  coordination  in  complex  ways.  For  example,  the  requirements  for 
communication  and  coordination  may  increase  when  computer-based  display  systems  and  decision  aids  do  not  make 
information  equally  accessible  to  all  members  of  the  crew.  Computer-based  HSI  technologies  can  also  interfere  with  the 
ability  of  members  to  monitor  each  other’s  actions  to  maintain  situation  awareness  and  monitor  for  errors.  Dien  and 
Montmayeul  (1995)  state  that  the  HSI  design  should  be  compatible  with  the  organizational  structure  of  the  crew  and  support 
manageable  workloads  while  ensuring  safety.  They  specifically  identify  as  topics  of  concern  the  use  of  HSI  design  to 
support  crew  coordination  and  the  flexible  allocation  of  tasks  between  crew  members.  This  first  concern  was  recognized  for 
such  diverse  items  as  operator-aiding  systems  in  NPPs  (Dien  and  Montmayeul,  1995),  group-view  displays  for  NPP  CRs 
(Stubler  and  O’Hara,  1996b),  computer-based  procedures  of  NPPs  (O’Hara,  Higgins,  Stubler,  and  Kramer,  2000),  and  ship 
bridge  design  (Hutchins,  1990).  However,  guidance  in  this  area  is  evolving  slowly,  and  more  research  is  needed  to  identify 
the  particular  dimensions  of  HSI  design  that  affect  the  crew’s  performance  and  the  degree  to  which  plant  performance  and 
safety  may  be  affected. 

The  Role  of  Training  in  HSI  Skills 

In  many  domains,  computer-based  HSI  technologies  provide  advanced  capabilities  for  presenting  and  processing 
information,  but  while  they  can  support  the  operator’s  information  processing  functions,  they  can  also  degrade  performance. 
For  example,  they  may  increase  an  operator’s  workload  by  creating  tasks  that  are  not  directly  related  to  operating  the  plant. 
They  also  may  create  control  and  display  configurations  that  increase  the  likelihood  of  errors  (Moray,  1992;  O’Hara, 


NUREG/CR-6637 


5-50 


5  TECHNICAL  BASIS  DEVELOPMENT:  DESIGN  PROCESS 


Stubler,  and  Nasta,  1997).  Reviews  of  industrial  experience  and  practices  suggest  that  operators  often  receive  little  or  no 
training  in  strategies  for  using  these  capabilities  effectively.  Training  in  using  cognitive  strategies  for  accessing 
information  from  the  HSl  can  be  an  effective  approach  for  supporting  high  levels  of  operator  performance  (Mumaw  et  al., 
1994),  but  such  skills  are  not  commonly  taught.  Performance  may  benefit  from  combined  training  that  address  both  the  use 
of  HSI  capabilities,  and  skills  and  strategies  for  making  the  best  use  of  information  processing  capabilities.  This  is  a 
growing  area  of  research  and  further  work  is  needed  to  assess  training  approaches  and  how  far  performance  may  be 
improved.  The  training  recommendations  provided  by  Mumaw  et  al.  (1994)  provide  some  initial  direction. 

The  Effects  of  the  Installation  Process  for  HSl  Upgrades  upon  Personnel  Performance 

Little  empirical  research  has  specifically  focused  on  how  human  performance  is  affected  by  installing  upgrades  in  an 
existing  HSl.  Upgrades  may  be  incorporated  in  many  different  ways.  In  the  simplest  case,  the  old  equipment  is  removed  at 
the  same  time  that  the  new  is  put  in.  However,  industrial  experience  indicates  that  the  old  HSI  components  are  often  not 
removed  immediately.  Instead,  a  gradual  transition  may  be  made  in  which  the  new  and  the  old  components  exist  together 
before  the  old  components  are  deactivated  and  removed.  In  other  cases,  both  the  old  and  new  components  may  remain 
present  and  functioning  in  the  HSI,  giving  the  user  the  choice  of  which  to  use.  In  addition,  when  plant  systems  are 
modified,  additional  instrumentation  may  be  temporarily  added  to  the  HSI  to  provide  special  control  and  display  capabilities 
that  reflect  the  temporary  configurations  of  plant  equipment. 

The  presence  of  old  non-functional  components  (i.e.,  abandoned  in  place)  represents  a  potential  source  of  clutter,  and  may 
distract  people’s  attention  during  monitoring  and  information  retrieval.  If  both  the  old  and  new  components  are  functional, 
there  may  be  problems  due  to  differences  in  methods  of  operating  them.  The  potential  problems  associated  with  alternating 
use  of  inconsistent  HSI  designs,  described  in  Section  5.3.1,  may  be  particularly  important  when  the  same  information  or 
control  capabilities  reside  in  devices  that  operate  differently.  For  example,  personnel  may  have  difficulty  recalling  the 
correct  method  for  interacting  with  one  device  if  the  other  was  used  recently.  Also,  strategies  for  using  the  devices,  such  as 
ways  for  rapidly  scanning  the  displayed  information  or  executing  control  actions  during  specific  operations,  may  not 
translate  well  between  devices  and,  therefore,  cause  errors. 

One  objective  of  HSI  task  support  verification  is  to  ensure  that  the  HSI  does  not  contain  information,  controls,  and  displays 
that  do  not  support  personnel  tasks.  However,  there  is  little  guidance  to  address  the  case  in  which  the  HSl  contains 
components  that  have  been  temporarily  or  permanently  deactivated.  The  general  approach  suggested  by  NUREG-071 1  is  to 
eliminate  such  HSI  components  and  information.  However,  no  specific  guidance  is  available  for  determining  when  this 
approach  is  not  desirable  or  needed.  Similarly,  guidance  is  sparse  for  cases  where  old  and  new  HSI  components  with 
different  methods  of  operation  exist  in  the  same  HSI,  as  it  is  for  instances  in  which  personnel  must  adapt  to  a  continually 
changing  HSl.  Lacking  detailed  guidance,  such  topics  may  be  evaluated  during  integrated  system  validation.  However, 
direct  research  should  be  undertaken  to  address  these  topics  more  directly. 

Personnel  Acceptance  of  Upgrades 

Researchers  have  suggested  that  personnel  acceptance  of  HSI  changes  can  be  enhanced  by  involving  users  in  their  design 
and  evaluation  (Medsker  and  Campion,  1997;  Swanekamp,  1995).  However,  experience  in  indushy  suggests  that  this  may 
not  always  be  completely  effective  or  sufficient.  Thus,  a  better  understanding  is  needed  of  the  causes  of  poor  operator 
acceptance,  its  impact  on  the  plant’s  operation  and  safety,  and  appropriate  countermeasures.  These  considerations  are 
important  both  for  the  transition  from  traditional  to  computer-based  technologies,  and  between  subsequent  more  complex 
upgrades  of  the  latter.  The  flexibility  of  computer-based  systems  and  the  rapid  pace  at  which  they  become  technologically 
obsolete  may  increase  the  rate  at  which  HSIs  are  upgraded  in  the  future.  Thus,  the  user’s  acceptance  of  changes  in  the  HSI 
will  be  a  growing  concern. 
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As  described  in  the  Section  1 .2,  the  review  of  upgrades  is  included  in  the  Standard  Review  Plan  (SRP)  (NUREG-OSOO). 

NUREG-0800  provides  guidance  to  NRC  staff  for  reviewing  NPPs  and  cites  NUREG-071 1  and  Part  1  of  NUREG-0700  for 

high-level  design  process  criteria  for  reviewing  overall  HFE  programmatic  goals  and  objectives.  Therefore,  guidelines  for 

HFE  reviews  of  upgrades  were  developed  according  to  the  ten  review  elements  of  NUREG-07 1 1 ;  they  are  presented  in 

Section  9.  They  re  organized  into  the  following  sections: 

•  General  Guidance 

•  HFE  Program  Management 

•  Operating  Experience  Review 

•  Functional  Requirements  Analysis  and  Function  Allocation 

•  Task  Analysis 

•  Staffing 

•  Human  Reliability  Analysis 

•  Human-System  Interface  Design 

•  Procedure  Development 

•  Training  Program  Development 

•  Human  Factors  Verification  and  Validation. 

Within  each  element,  the  guidance  is  further  organized  into  the  following  four  categories: 

•  Guidance  for  the  Applicability  of  the  NUREG-07 11  Element  -  This  category  includes  guidance  defining  when  and 
how  an  element  of  NUREG-071 1  is  applicable  to  reviewing  an  upgrade.  It  is  found  at  the  beginning  of  each 
section  addressing  a  NUREG-071 1  element,  and  states  the  following:  the  conditions  under  which  NUREG-071 1 
element  is  applicable  to  an  upgrade,  the  section  of  that  document  which  covers  the  element,  and  overall  scope  of 
the  NUREG-071 1  element  as  it  pertains  to  upgrades. 

•  Guidance  Tailored  from  NUREG-071 1  for  Upgrades  -  This  category  includes  modified  guidance  from  NUREG- 
071  1  that  focuses  on  characteristics  and  considerations  relevant  to  upgrades.  Because  NUREG-071 1’s  scope  is 
broad,  reviewers  may  have  difficulty  interpreting  the  guidelines  in  the  context  of  upgrades.  This  category  focuses 
the  guidance  of  NUREG-071 1  on  specific  aspects  of  upgrades  that  may  be  important  to  human  performance  and 
safety.  However,  this  category  does  not  repeat  those  guidelines  from  NUREG-071 1  that  can  be  directly  applied  to 
upgrades  without  special  interpretation. 

•  New  Guidance  for  Upgrades  Only  -  This  category  includes  guidance  specifically  relevant  to  upgrades  that  does  not 
currently  appear  in  NUREG-071 1 . 
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•  New  Guidance  for  the  NUREG-0711  Element  -  The  development  of  review  guidance  for  upgrades  identified  some 

considerations  with  broad  applicability.  They  are  suggested  as  possible  additions  to  the  more  general  guidance  of 
NUREG-0711. 

When  reviewing  an  upgrade,  the  NRC  reviewer  should  first  follow  NUREG-071 1  and  then  consider  the  guidance  in 
Section  9  which  focuses  on  special  concerns  for  upgrades. 
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7  SUMMARY 


The  objective  of  this  study  was  to  formulate  HFE  review  guidance  for  NPP  upgrades  to  plant  systems  and  the  HSI,  based  on 
a  technically  valid  methodology  for  developing  guidelines.  Several  tasks  were  performed,  including  the  following: 

•  A  technical  basis  was  established  using  human  performance  research  and  analyses  of  upgrades, 

•  HFE  review  guidelines  were  developed  in  a  format  consistent  with  NUREG-071 1,  and  other  NRC  guidance,  and 

•  Remaining  issues  were  identified  for  which  research  is  insufficient  to  support  the  development  of  NRC  review 
guidance. 

The  status  of  each  is  briefly  discussed  below. 

Technical  Basis  Development 

The  effects  of  upgrades  on  personnel  performance  were  addressed  by  examining  basic  HFE  literature,  literature  on  complex 
human-machine  systems,  and  industry  experience  gained  from  site  visits,  interviews,  and  industrial  literature.  The 
resulting  technical  basis  was  described  in  two  sections.  Section  4  reviewed  human  performance  in  complex  human- 
machine  systems,  and  discussed  the  types  of  knowledge  and  skills  that  must  be  adapted  to  a  new  work  design  after  an 
upgrade.  Section  5  was  a  more  detailed  discussion  of  these  considerations  within  the  context  of  the  NUREG-071 1  review 
process.  Specific  considerations  were  identified  for  elements  of  the  NUREG-071 1  review. 

HFE  Review  Guidelines 


Guidance  for  reviewing  upgrades  was  developed  to  address  the  design  process.  The  SRP  specifies  NUREG-071 1  as  the 
process  used  for  HFE  reviews  of  design  processes.  Therefore,  guidance  for  upgrades  was  organized  in  Section  9  according 
to  the  ten  review  elements  of  NUREG-071 1.  Within  each  element,  guidance  was  further  organized  into  four  sections.  The 
first  describes  the  conditions  under  which  the  particular  NUREG-071 1  element  is  relevant  to  the  review  of  upgrades.  The 
second  category  includes  modified  guidance  from  NUREG-071 1  that  focuses  on  characteristics  and  considerations  for 
upgrades.  The  guidance  in  the  third  category  is  specifically  relevant  to  upgrades,  but  does  not  appear  in  NUREG-071 1 . 

The  fourth  category  has  considerations  that  have  potential  applications  beyond  upgrades,  and  are  possible  additions  to  the 
more  general  guidance  of  NUREG-071 1. 

Upgrade  Issues  Requiring  Additional  Research 

Several  human  performance  issues  associated  with  upgrades  were  identified  in  Section  5.6.  They  represent  topics  for  which 
research  is  necessary  before  more  guidance  can  be  developed: 

•  The  Effects  of  HSI  Inconsistency  on  Alternating  Use  of  HSI  Components 

•  The  Effects  of  HSI  Design  on  Crew  Coordination  and  Cooperation 

•  The  Role  of  Training  in  HSI  Skills 

•  The  Effects  of  the  Installation  Process  for  HSI  Upgrades  on  Personnel  Performance 

•  Personnel  Acceptance  of  Upgrades. 
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Review  Guidelines:  Hybrid  HSI  Design  Process 


9  DESIGN  PROCESS  REVIEW  GUIDANCE  FOR  HYBRID  HSI 


The  guidelines  for  the  design  process  were  developed  to  address  important  considerations  identified  in  the  literature  and  to 
provide  a  means  whereby  human  performance  issues  may  be  assessed  during  a  design  review  (see  Section  6).  The  review 
guidelines  were  formatted  to  correspond  to  the  NRC’s  general  design  process  guidance  in  NUREG-071 1 .  They  are 
organized  into  the  following  sections: 

•  General  Guidance 

•  HFE  Program  Management 

•  Operating  Experience  Review 

•  Functional  Requirements  Analysis  and  Function  Allocation 

•  Task  Analysis 

•  Staffing 

•  Human  Reliability  Analysis 

•  Human-System  Interface  Design 

•  Procedure  Development 

•  Training  Program  Development 

•  Human  Factors  Verification  and  Validation. 

Some  guidelines  specify  that  particular  aspects  of  an  upgrade  should  be  “evaluated.”  General  approaches  to  evaluation  and 
review  criteria  are  defined  in  NUREG-  0700;  such  evaluations  are  not  described  here. 

9.1  General  Guidance 

( 1 )  An  HFE  review  should  be  conducted  if  the  upgrade^  of  a  plant  system  or  the  HSI  affects  the  role  of  personnel  or  the 
tasks  by  which  their  role  is  performed  and  is  potentially  significant  to  plant  safety.  Upgrades  affect  the  role  or 
tasks  of  personnel  if  they  impose  new  or  different  demands  on  them  to  operate,  maintain,  or  otherwise  ensure 
safety.  An  upgrade  may  be  considered  potentially  significant  to  plant  safety  if  it 

•  Constitutes  an  unreviewed  safety  question,  as  defined  in  10  CFR  50.59,  or 

•  Isa  modification  of  a  structure,  system,  or  component  (SSC)  that  is  safety  related,  or 

•  Isa  modification  of  a  non-safety  SSC  that  (a)  mitigates  accidents  or  transients,  (b)  is  used  in  emergency 
operating  procedures,  or  (c)  could  prevent  a  safety-related  SSC  from  fulfilling  its  function. 


^The  term  upgrade  is  used  generically  to  include  any  type  of  change,  modification,  or  retrofit  made  to  HSI 
components  or  plant  systems  that  may  influence  personnel  performance. 
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The  goal  of  HFE  reviews  is  to  ensure  plant  safety.  An  upgrade  may  constitute  an  unreviewed  safety  question  (10 
CFR  50.59,  EPRI,  1993)  if  it  may 

•  increase  the  probability  of  an  accident  evaluated  in  the  Safety  Analysis  Report  (SAR) 

•  increase  the  consequences  of  an  accident  evaluated  in  the  SAR 

•  increase  the  probability  of  a  malfunction  of  equipment  important  to  safety  evaluated  in  the  SAR 

•  increase  the  consequences  of  a  malfunction  of  equipment  important  to  safety  evaluated  in  the  SAR 

•  create  the  possibility  of  an  accident  of  a  different  type  than  any  evaluated  in  the  SAR 

•  create  the  possibility  of  a  malfunction  of  equipment  important  to  safety  when  such  a  malfunction  differs 
from  any  evaluated  previously  in  the  SAR 

•  reduce  the  margin  of  safety  as  defined  in  the  basis  for  any  technical  specification. 

Discussion:  The  criteria  covering  structures,  systems,  or  components  were  adapted  from  the  Maintenance  Rule, 
NRC  Inspection  Procedure  62706  (NRC,  1995e). 

9.2  HFE  Program  Management 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(1)  The  HFE  program  management  review  should  be  conducted,  in  accordance  with  Section  2  of  NUREG-071 1,  for 
upgrades  identified  in  Section  9. 1 .  The  licensee  should  prepare  an  HFE  program,  which  is  a  technical  plein 
describing  how  HFE  considerations  will  be  addressed  in  the  design,  development,  and  implementation  of  the 
upgrade.  The  scope  of  the  HFE  program  should  be  consistent  with  the  scope  of  the  upgrade. 

Discussion:  See  NUREG-07 1 1 ,  Section  2. 

Guidance  Tailored  from  NUREG-07 11  for  Upgrades 

(2)  The  HFE  program  should  ensure  that  the  HFE  requirements  are  specified  in  agreements  with  vendors  or 
subcontractors,  who  will  be  providing  upgrades,  as  described  in  Criterion  6  of  Section  2.4.3  of  NUREG-071 1. 
These  requirements  should  ensure  that  new  components  adhere  to  the  principfes  of  good  HFE  design,  including 
compatibility  with  the  existing  HSI.  Criteria  for  selecting  off-the-shelf  equipment  should  include  human  factors 
requirements.  If  the  upgrade  will  involve  acquiring  equipment  that  may  contain  human  factors  discrepancies,  the 
HFE  program  should  ensure  that  analyses  and  evaluations  will  identify  potential  problems,  assess  their  potential 
importance  to  plant  safety,  and  identify  and  implement  solutions  for  ensuring  plant  safety. 

Discussion:  This  criterion  is  important  for  upgrades  since  add-ons  or  system  modifications  often  are  purchased  as 
modular  units  designed  to  the  standard  specifications  of  the  vendor,  rather  than  the  plant’s  specifications.  In  cases 
involving  off-the-shelf  components,  factors  such  as  the  availability  of  vendors,  technological  compatibility,  and 
constraints  on  proprietary  information,  may  necessitate  obtaining  equipment  that  has  human  factors  discrepancies; 
the  HFE  program  should  address  them. 


NUREG/CR-6637 


9-2 


9  DESIGN  PROCESS  REVIEW  GUIDANCE  FOR  HYBRID  HSI 


New  Guidance  for  Upgrades  Only 

(3)  The  HFE  program  should  ensure  that  the  knowledge  and  experience  of  plant  personnel,  who  will  use  the  upgrade, 
will  be  incorporated  in  developing  and  implementing  the  upgrade.  The  development  may  include  designing  new 
components  or  selecting  off-the-shelf  ones.  The  HFE  program  should  involve  plant  personnel  to  ensure  that  the 
following  are  considered  from  a  user’s  perspective  in  establishing  upgrade  requirements  and  evaluating  the  results 
of  the  design  process: 

•  User’s  understanding  (mental  model)  of  how  plant  systems  are  structured  and  behave 

•  Task  dem2mds  and  constraints  of  the  existing  work  environment 

•  Existing  work  processes 

•  Organizational  goals  that  affect  the  implementation  and  use  of  the  upgrade. 

Discussion:  Plant  personnel  who  will  be  the  ultimate  users  of  the  upgrade  can  be  valuable  sources  of  domain- 
specific  information  that  should  be  considered  when  developing  and  evaluating  upgrades  (e.g.,  the  design 
requirements  and  evaluation  criteria).  Their  involvement  can  help  ensure  that  the  final  design  of  the  upgrade  is 
consistent  with  organizational  concerns,  such  as  reliability,  maintainability,  and  supportability  (Medsker  and 
Campion,  1997;  Price,  1990).  (See  Sections  5.1  and  5.5.) 

(4)  The  goals  of  the  HFE  program  should  address  the  need  to  consider  the  effects  that  the  upgrade  may  have  on  the 
performance  of  personnel.  The  transition  from  the  existing  plant  configuration  to  the  upgrade  configuration 
should  be  planned  so  that  adapting  to  the  change  imposes  minimal  demands.  The  considerations  should  include 
the  following: 

•  Planning  the  installation  to  minimize  disruptions  to  ongoing  operations 

•  Coordinating  changes  to  training  materials  and  procedures  to  ensure  that  they  accurately  reflect  the 
systems  being  upgraded 

•  Conducting  training  to  maximize  personnel’s  knowledge  and  skill  with  the  new  design  before  its 
implementation. 

Discussion:  Industrial  experience  showed  that  temporary  HSI  and  plant  configurations  that  occur  when  upgrades 
are  established  over  a  period  can  pose  demands  on  human  performance  that  differ  from  either  the  initial  or  final 
configurations.  (See  Section  5.4.) 

New  Guidance  for  the  NUREG-07JJ  Element 

(5)  HFE  considerations  identified  when  implementing  a  new  or  upgraded  design  of  plant  systems  or  HSI  components, 
or  during  its  subsequent  operation,  should  be  recorded  in  an  HFE  issues  tracking  system  as  described  in  Section 
2.4.4  ofNUREG-0711. 

Discussion:  HFE  issues  not  detected  during  HFE  verification  and  validation  may  become  apparent  during  plant 
operation.  The  criteria  in  Section  2.4.4  of  NUREG-071 1  should  be  expanded  to  address  subsequent  issues. 
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9.3  Operating  Experience  Review 

Guidance  for  the  Applicability  of  the  NUREG-071 1  Element 

(1)  This  review  should  be  conducted  in  accordance  with  Section  3  of  NUREG-071 1  for  all  upgrades  identified  in 
Section  9. 1 .  The  scope  of  the  operating  experience  review  (OER)  should  emphasize  the  systems  or  HSI  components 
that  are  being  modified. 

Discussion:  The  OER  should  be  focused  to  provide  information  relevant  to  the  upgrade. 

Guidance  Tailored from  NUREG-07JJ  for  Upgrades 

(2)  The  OER,  as  described  in  Section  3.4.1  of  NUREG-071 1,  should  address  the  operating  experience  of  the  plant  that 
will  be  upgraded,  including  experiences  with  the  systems  that  will  be  upgraded  and  with  HSI  technologies  that  are 
similar  to  those  under  consideration  for  the  upgrade.  Also,  when  operators  and  maintenance  personnel  are 
unfamiliar  with  the  proposed  technology,  attention  should  be  paid  to  the  operating  experience  of  other  plants  that 
already  have  the  technology. 

Discussion:  This  guideline  is  derived  from  the  criteria  of  Section  3.4.1  of  NUREG-071 1,  and  focuses  on 
considerations  related  to  upgrading  existing  plants. 

New  Guidance  for  the  NUREG‘0711  Element 

(3)  The  OER  should  identify  risk-important  tasks  that  have  been  prone  to  errors.  These  tasks  should  receive  special 
attention  during  the  design  of  the  user  interface  to  lessen  their  probability. 

Discussion:  When  changing  the  user  interface  from  analog  to  digital  technology,  the  likelihood  of  some  types  of 
errors  may  be  increased.  Where  their  potential  consequences  are  high,  design  approaches  should  be  used  to  protect 
against  them  (Stubler,  O’Hara,  and  Kramer,  2000). 

9.4  Functional  Requirements  Analysis  and  Function  Allocation 

Guidance  for  the  Applicability  of  the  NUREG-071]  Element 

(1)  Functional  requirements  analyses  should  be  reviewed  in  accordance  with  Section  4  of  NUREG-071 1,  for  all 
upgrades  identified  in  Section  9.1  that  are  likely  to  change  existing  functions  that  are  important  to  safety,  introduce 
new  functions  for  systems  supporting  safety  functions,  or  involve  unclear  functional  requirements  that  may  be 
important  to  safety.  The  functional  requirements  analysis  should  address  new  functions  resulting  from  changes  in 
the  degree  of  integration  between  plant  systems.  For  example,  installing  higher-level  automation  may  bring 
systems  that  were  formerly  controlled  separately  under  a  single  controller.  Also,  the  upgrades  may  change  the 
degree  to  which  different  plant  systems  share  common  resources  (e.g.,  power  sources,  cooling  water,  and  data- 
transmission  buses).  These  may  be  important  in  diagnosing  malfunctions  or  planning  responses.  The  functional 
requirements  analyses  should  be  revised  and  updated  to  reflect  requirements  of  the  upgrade;  the  scope  of  the 
analyses  may  be  restricted  to  functions  related  to  the  upgrade. 

Discussion:  Upgrades,  such  as  increased  automation  of  systems,  may  change  the  role  of  personnel.  For  example, 
they  may  create  new  functions,  such  as  detecting  malfunctions  of  the  new  systems.  (See  Sections  5.2.1  and  4.4.) 

(2)  Function  allocation  analyses  should  be  reviewed,  in  accordance  with  Section  4  of  NUREG-071 1,  for  all  upgrades 
identified  in  Section  9.1  that  are  likely  to  change  the  allocation  between  personnel  and  plant  systems  of  functions 
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important  to  safety.  The  analyses  should  be  revised  and  updated  to  reflect  requirements  for  the  upgrade;  their 
scope  may  be  restricted  to  functions  involving  the  upgrade. 

Discussion:  Upgrades,  such  as  increased  levels  of  automation  in  plant  systems,  may  change  the  allocation  of 
control  functions  between  the  plant’s  systems  and  operators.  The  operator’s  role  in  controlling  systems  manually 
may  be  more  demanding  when  it  is  performed  as  a  backup  to  an  automated  system  than  when  the  systems  are 
always  manually  controlled.  (See  Sections  5.2.1  and  4.4.) 

Guidance  Tailored  from  NUREG-07 1 1  for  Upgrades 

(3)  A  change  in  an  operator’s  role  due  to  an  upgrade  should  be  examined  within  the  context  of  its  effects  on  the 

operator’s  overall  responsibilities.  Increases  in  certain  task  demands  may  affect  the  ability  of  the  operator  to  carry 
out  others  that  are  risk  important. 

Discussion:  This  guideline  is  adapted  from  NUREG-07 1 1 . 

9.5  Task  Analysis 

Guidance  for  the  Applicability  of  the  NUREG-07 11  Element 

(1)  This  review  should  be  conducted,  in  accordance  with  Section  5  of  NUREG-071 1,  for  all  upgrades  identified  in 
Section  9.1  that  are  likely  to  affect  tasks  previously  identified  as  risk  important,  cause  existing  ones  to  become  risk 
important,  or  create  new  tasks  that  are  risk  important.  The  task  analyses  should  be  revised  and  updated  to  reflect 
requirements  of  the  upgrade;  the  scope  should  include  tasks  involving  the  upgrade  and  its  interactions  with  the  rest 
of  the  plant,  including  those  resulting  from  functions  addressed  in  the  analyses  of  functional  requirements  and 
function  allocation.  For  maintenance,  tests,  inspections,  and  surveillances,  attention  should  be  given  to  risk- 
important  tasks  that  are  new  or  supported  by  new  technologies  (e.g.,  new  capabilities  for  on-line  maintenance). 
Discussion:  A  task  analysis  may  not  be  necessary  if  people’s  tasks  do  not  change.  (See  Section  5.2.2.)  New 
capabilities  for  maintenance  are  described  in  Stabler,  Higgins,  and  Kramer  (2000). 

Guidance  Tailored from  NUREG-07 11  for  Upgrades 

(2)  Task  analyses  should  identify  user  strategies  for  processing  information  and  executing  actions  that  are  needed  to 
perform  successfully  over  the  anticipated  range  of  workload.  Strategies  identified  in  the  existing  HSI  should  be 
examined  to  understand  task  demands  under  the  current  design.  These  demands  should  be  taken  into  account 
when  developing  design  requirements  for  the  upgrade. 

Discussion:  Task  analysis  methods  that  only  focus  on  observable  actions  may  ignore  cognitive  skills  important  to 
skilled  performance  in  highly  demanding  situations  (e.g.,  high  time  pressure,  high  information  content,  and 
rapidly  changing  conditions)  (Mumaw  et  al.,  1994;  Klein  et  al.,  1989).  The  failure  to  consider  cognitive  skills 
required  for  a  job  may  result  in  simplistic  assumptions  about  personnel’s  tasks  and  the  development  of  inadequate 
design  requirements.  (For  more  information,  see  Changes  in  Personnel  Tasks  and  Cognitive  Task  Analysis  in 
Section  5.2.2.) 

(3)  Routine  tasks,  which  may  benefit  from  the  ability  to  undertake  actions  without  highly  focused  attention,  and  tasks 
requiring  similar  actions  should  be  identified  so  they  may  be  addressed  consistently  when  designing  the  user 
interface.  In  this  way,  similar  user  interfaces  may  be  provided  for  similar  actions. 

Discussion:  The  ability  to  act  without  paying  close  attention  can  enhance  a  person’s  performance  by  freeing 
mental  resources  for  higher-level  activities  (Schneider,  1985;  Mumaw  and  Gabrys,  1996;  Schlager  et  al.,  1989). 
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Overall  performance  may  suffer  if  the  new  upgrade  does  not  support  automaticity  for  low-level  tasks  but,  instead, 
increases  task  demands.  (For  additional  information,  see  Performing  Tasks  without  Highly  Focused  Attention  in 
Section  5.2.2  and  Automaticity  of  Action  in  Section  4.3.2.) 

New  Guidance  for  Upgrades  Only 

(4)  The  task  analysis  should  identify  the  design  characteristics  of  the  existing  HSI  that  support  the  superior 
performance  of  experienced  personnel  (e.g.,  support  high  levels  of  performance  during  demanding  situations). 

Such  characteristics,  which  may  include  the  spatial  arrangement  of  control  and  display  devices  and  the  ability  to 
adjust  controls  and  displays  to  deal  with  special  tasks,  should  be  considered  in  developing  new  design 
requirements.  The  new  design  should  have  features  performing  similar  functions,  or  should  eliminate  the  need  for 
them  by  performing  these  functions  differently.  In  addition,  the  task  analysis  should  identify  and  examine 
adjustments  made  to  the  HSI  by  users,  such  as  notes  and  external  memory  aids,  which  suggest  the  users’  needs  may 
not  be  fully  met  by  its  current  design.  All  task  demands  should  be  adequately  addressed  by  the  new  design 
requirements.  Design  features  identified  during  OERs  should  be  considered  in  these  analyses. 

Discussion:  HSI  adjustments  made  by  users  to  fulfil  task  demands  may  highlight  aspects  of  their  tasks  that  are 
particularly  challenging  and  may  not  be  fully  supported  by  the  design  of  the  HSI  (Schlager  et  al.,  1989).  (For 
additional  information,  see  HSI  Tailoring  in  Section  5.2.2.) 

New  Guidance  for  the  NUREG-0711  Element 

(5)  The  task  analysis  should  identify  tasks  especially  prone  to  errors.  This  analysis  should  include  tasks  identified 
during  the  OER  as  being  associated  with  errors.  Error  prone  tasks  should  be  designated  as  requiring  special 
attention  during  the  design  of  the  user  interface  to  protect  against  errors. 

Discussion:  Computer-based  user  interfaces  may  be  especially  prone  to  certain  types  of  errors.  When  changing 
the  user  interface  to  a  computer-based  technology,  the  likelihood  of  some  types  of  errors  may  increase.  Where  the 
potential  consequences  of  such  errors  are  high,  design  approaches  should  be  used  to  protect  against  them  (Stubler, 
O’Hara,  and  Kramer,  2000). 

9.6  Staffing 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(1)  This  review  should  be  conducted  in  accordance  with  Section  6  of  NUREG-071 1  for  all  upgrades  identified  in 
Section  9.1  that  are  likely  to  change  staffing  requirements.  The  scope  of  the  analysis  should  address  staffing 
demands  that  result  from  the  upgrade  and  its  interactions  with  the  rest  of  the  plant.  An  upgrade  may  be  likely  to 
change  staffing  requirements  if  it  does  any  of  the  following: 

•  Changes  the  roles  of  operators,  as  determined  by  the  function  analysis 

•  Decreases  the  availability  of  operators  (e.g.,  increases  demands  for  tasks  outside  of  the  CR) 

•  Decreases  the  ability  to  coordinate  work 

•  Increases  demands  for  operators  to  coordinate  their  work 
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•  Decreases  the  availability  or  accessibility  of  information  needed  by  operators 

•  Changes  the  levels  of  knowledge,  skill,  and  experience  from  those  specified  by  current  qualification 
requirements. 

Discussion:  Changes  in  plant  systems  or  the  HSI  may  impose  new  requirements  for  interactions  between  CR 
personnel  or  for  personnel  knowledge  and  skills  which  exceed  the  assumptions  used  to  establish  current  staffing 
levels  (Hutchins,  1990).  (For  additional  information,  see  Section  5.2.3.) 

9.7  Human  Reliability  Analysis 

Guidance  for  the  Applicability  of  the  NUREG-071 1  Element 

(1 )  This  review  should  be  conducted  in  accordance  with  Section  7  of  NUREG-07 1 1  for  all  upgrades  identified  in 
Section  9. 1  that  may  affect  tasks  previously  identified  as  risk  important,  cause  existing  tasks  to  become  risk 
important,  or  create  new  ones  that  are  risk  important.  The  scope  of  the  human  reliability  analysis  should  address 
personnel  actions  resulting  from  the  upgrade  and  its  interactions  with  the  rest  of  the  plant. 

Discussion:  Changes  in  the  HSI  or  plant  systems  may  change  the  risk  importance  of  tasks,  so  that  conclusions 
drawn  from  previous  HRAs  may  be  inaccurate. 

Guidance  Tailored  from  NUREG-07 11  for  Upgrades 

(2)  When  upgrading  plant  systems  or  the  HSI,  consideration  should  be  given  to  the  following  effects  of  these 
modifications  on  the  existing  HRA: 

•  Whether  the  original  HRA  assumptions  are  valid  for  the  upgraded  design 

•  Whether  the  human  errors  analyzed  in  the  existing  HRA  are  still  relevant 

•  Whether  the  probability  of  errors  by  operators  and  maintenance  personnel  may  change 

•  Whether  errors  may  be  introduced  that  are  not  modeled  by  the  existing  HRA  and  PRA 

•  Whether  the  consequences  of  errors,  established  in  the  existing  HRA,  may  change. 

Discussion:  HRAs  often  are  based  on  assumptions  about  operators’  ability  to  detect  a  fault  and  begin  a  manual 
response  within  a  specific  time.  Changes  to  plant  systems  may  alter  the  bases  of  these  assumptions,  and  affect  each 
of  the  considerations  listed  above.  For  example,  introducing  new  levels  of  automation  may  meike  the  operator’s 
role  of  detecting  and  responding  to  malfunctions  more  challenging.  Changes  to  the  HSI  may  lessen  the  operators’ 
ability  to  access  information  and  controls  quickly  and  accurately  (O’Hara,  Stubler  and  Nasta,  1997).  In  addition, 
changes  to  systems  and  the  HSI  may  affect  their  ability  to  carry  out  maintenance  and  operations  correctly  (Stubler, 
O’Hara,  and  Kramer,  2000;  Stubler,  Higgins,  and  Kramer,  1998). 

(3)  If  the  upgrade  involves  new  or  existing  human  actions  that  are  risk  important,  then  these  actions  should  be  re¬ 
examined  as  described  in  Criterion  3  of  Section  7.4  of  NUREG-071 1,  and  addressed  accordingly. 

Discussion:  Section  7.4  of  NUREG-071 1  gives  criteria  for  reviewing  HRAs. 
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9.8  Human-System  Interface  Design 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(1)  All  upgrades  identified  in  Section  9. 1  should  be  reviewed,  in  accordance  with  Section  8  of  NUREG-071 1  when 
they  change  existing  HSI  components  used  for  risk-important  tasks,  or  add  new  HSI  components  to  risk-important 
tasks.  TTie  scope,  specified  in  Section  8.4. 1 . 1  NUREG-07 1 1 ,  may  be  limited  to  the  HSI  upgrade  and  its 
interactions  with  the  other  components  of  the  HSI.  Inputs  to  the  design  process  specified  in  Section  8.4. 1.2  may  be 
limited  to  those  from  the  HSI  design  process  analyses  that  involve  the  upgrade.  The  HSI  design  procedures, 
specified  in  Section  8.4. 1.3,  may  be  limited  to  those  analyses  that  apply  to  the  upgrade. 

Discussion:  An  HSI  design  review  should  address  risk- important  aspects  of  the  upgrade  (NUREG-071 1). 

9.8.1  HSI  Design  Methodology 

9.8.1. 1  Development  of  HSI  Design  Guidance 

Guidance  Tailored from  NUREG-0711  for  Upgrades 

(2)  The  development  of  guidance  for  interface  design  should  include  upgrades.  This  guidance  should  specifically 
address  consistency  in  design  across  the  HSI.  As  far  as  possible,  it  should  specify  standard  formats  for  controls  and 
displays  that  are  used  for  categories  of  commonly  occurring  control  actions.  The  failure  to  develop  such  standard 
formats  should  be  justified. 

Discussion:  This  guideline  is  derived  from  the  guidance  for  tailoring  the  HFE  design  guidelines  in  Section  8.4.2. 1 
of  NUREG-071 1,  and  also  the  High-Level  Design  Review  Principle  of  Consistency  (see  Appendix  A). 
Investigations  of  computer-based  HSIs  have  shown  that  many  types  of  errors  may  be  prevented  or  eliminated  by 
employing  design  guidance  that  encourages  the  consistent  use  of  sound  HFE  principles  across  the  HSI  (i.e., 

Stubler,  O’Hara,  and  Kramer,  2000). 

9.8.1.2  HSI  Detailed  Design  and  Integration 

New  Guidance  for  Upgrades  Only 

(3)  HSI  upgrades  should  be  designed,  to  the  extent  possible,  to  be  consistent  with  users’  existing  strategies  for 
gathering  and  processing  information  and  executing  actions,  as  identified  in  the  task  analysis.  Consistency  with 
existing  strategies  can  reduce  the  learning  required  for  personnel  to  become  proficient  in  using  the  upgrade. 
Introducing  designs  that  require  personnel  to  develop  new  strategies  should  be  avoided,  unless  the  new  upgrade  has 
significant  benefits  (i.e.,  the  users  should  not  be  required  to  abandon  existing  skills  and  acquire  new  ones  unless 
the  new  methods  provide  distinct  benefits,  such  as  allowing  the  user  to  reach  a  higher  level  of  performance). 
Discussion:  The  design  of  conventional  (e.g.,  analog,  hardwired)  HSIs  supports  high  levels  of  personnel 
performance  in  ways  that  are  not  always  obvious;  experienced  operators  often  have  special  strategies  for  extracting 
information  from  them  (Schlager  et  al.,  1989).  The  failure  to  consider  the  strategies  and  HSI  characteristics  used 

in  the  existing  environment  may  result  in  simplistic  or  inadequate  design  requirements  for  an  upgrade.  (For 
additional  information,  see  Design  for  Consistency  Between  New  HSI  Components  and  the  Rest  of  the  HSI,  in 
Section  5.3.1,  Basic  Properties  of  Human  Information  Processing  and  Skilled  Performance  in  Section  4.3,  and 
Generic  Cognitive  Tasks  in  Section  4.4.) 
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(12)  Design  requirements  for  computer-based  HSI  upgrades  should  include  requirements  for  crew  coordination  and 

define  design  characteristics  for  supporting  it.  The  requirements  should  be  derived  from  function  and  task  analyses 
and  should  include  communication  and  flexible  allocation  of  tasks  between  crew  members.  If  the  proposed  design 
may  limit  crew  coordination,  the  design  requirements  should  include  features  for  overcoming  these  potential 
problems.  Design  characteristics  that  may  limit  crew  coordination  include  features  that  limit  the  ability  of 
personnel  to  have  a  shared  view  of  plant  information  (e.g.,  decision  aids  and  display  devices  that  can  only  be 
accessed  by  one  individual),  maintain  an  awareness  of  others’  actions,  and  communicate  effectively  with  others 
from  anticipated  work  locations.  The  design  requirements  should  ensure  that  the  HSI  is  compatible  with  the 
organizational  structure  of  the  crew,  and  produces  manageable  workload  while  ensuring  plant  safety. 

Discussion:  Introducing  computer-based  HSI  technologies  may  entail  complex  interactions  with  crew  coordination 
and  have  negative  effects  (Hutchins,  1990;  Stubler  and  O’Hara,  1996b;  O’Hara,  Higgins,  Stubler,  and  Kramer, 
2000).  This  guideline  is  an  application  of  the  High-Level  Design  Review  Principle  of  Task  Compatibility  (see 
Appendix  A).  (For  additional  information,  see  Changes  in  Personnel  Tasks  at  the  Crew  Level  in  Section  5.2.2  and 
Crew  Performance  and  Team  Skills  in  Section  4.4.2.  Additional  organizational  factors  are  discussed  in  Section 
5.5.) 

(5)  If  the  degree  of  integration  between  plant  systems  is  changed,  then  design  requirements  should  be  developed  to 
ensure  that  the  HSI  supports  personnel  in  controlling  these  systems.  For  example,  higher-level  automation  may 
bring  together  under  a  single  controller  systems  that  were  formerly  controlled  separately.  Also,  system  upgrades 
may  change  the  degree  to  which  different  systems  share  common  resources  (e.g.,  power  sources,  cooling  water,  and 
data-transmission  buses).  In  addition,  the  design  requirements  should  ensure  that  the  HSI  does  not  suggest  a 
degree  of  integration  between  plant  systems  that  does  not  actually  exist  (e.g.,  the  ability  to  access  a  set  of  plant 
variables  from  the  same  control  or  display  device  may  suggest  to  the  operator  that  they  are  functionally  related 
when  they  are  not).  The  design  requirements  of  the  HSI  should  ensure  that  the  relationships  between  plant  systems 
are  clearly  and  accurately  depicted. 

Discussion:  Industrial  experience  has  shown  that  the  design  of  computer-based  HSIs  can  lead  to 
misunderstandings  about  the  integration  between  plant  systems.  This  guideline  is  an  application  of  the  High-Level 
Design  Review  Principle  of  Task  Compatibility  (see  Appendix  A). 

Ne^  Guidance  for  the  NUREG-071 1  Element 

(6)  The  detailed  design  of  the  HSI  should  be  consistent  with  the  interface  design  guidance,  such  as  in  its  use  of 
standard  formats.  Any  discrepancies  should  be  justified  based  on  HFE  design  process  tests  and  evaluations. 
Discussion:  A  purpose  of  the  HSI  design  guidance  described  in  Section  8.4.2. 1  of  NUREG-071 1  is  to  ensure 
adherence  to  sound  HFE  principles,  including  consistency  of  design  characteristics  across  the  HSI.  Therefore, 
deviations  from  this  guidance  should  be  justified. 

9.8.2  HSI  Design  Process  Tests  and  Evaluations 

New  Guidance  for  Upgrades  Only 

(7)  When  an  HSI  component  is  being  replaced  with  an  upgrade,  evaluations  should  be  performed  to  assess  the 
consistency  between  the  methods  of  operating  the  new  and  the  old  HSI  component.  New  methods  should  be 
considered  unacceptable  if  prior  experience  with  former  methods  suggest  they  will  increase  the  likelihood  of  errors 
or  increase  mental  workload  required  to  avoid  them.  New  HSI  components  that  are  unacceptable  should  be 
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rectified  in  the  HSI  design  and  implementation  process  through  modifications  to  improve  HSI  compatibility, 
personnel  training,  and  procedure  development. 

Discussion:  This  guideline  addresses  Section  8.4.3.2  of  NUREG-071 1  and  is  consistent  with  the  High-Level 
Design  Review  Principle  of  Response  Workload  (see  Appendix  A).  It  is  also  consistent  with  the  principle  of 
transfer  of  training  (Holding,  1987;  Swezey  and  Llaneras,  1997;  Sawyer,  Pain,  Van  Cott,  and  Banks,  1982). 

(8)  Assessments  should  be  made  of  the  level  of  workload  and  the  degree  of  consistency  between  the  ways  of  operating 
the  new  HSI  components,  and  between  the  new  HSI  components  and  the  other  components  of  the  HSI.  These 
evaluations  should  include  the  alternating  use  of  different  interfaces,  particularly  where  the  same  user  is  expected 

to  use  different  devices  with  overlapping  methods  of  operation.  New  methods  should  be  considered  unacceptable  if 
alternating  use  may  increase  the  likelihood  of  errors  or  increase  the  mental  workload  required  to  avoid  them.  In 
addition,  the  level  of  workload  associated  with  using  the  upgraded  HSI  should  be  investigated  through 
performance-based  measures  and  interviews,  which  should  identify  strategies  users  employ  in  managing  workload. 
For  example,  complex  strategies  may  indicate  that  the  HSI  design  is  not  highly  compatible  with  human 
capabilities.  Unacceptable  new  components  should  be  addressed  through  the  HSI  design  and  implementation 
process  to  improve  their  compatibility,  or  the  likelihood  of  human  problems  reduced  through  other  means,  such  as 
training  and  procedure  development. 

Discussion:  This  guideline  addresses  Section  8.4.3.2  of  NUREG-071 1.  Empirical  studies  showed  that  human 
performance  may  show  a  greater  decrement  when  different  devices  have  overlapping  methods  of  operation  than 
when  they  are  very  different  (Tanaka  et  al.,  1991).  This  guideline  is  consistent  with  the  High-Level  Design  Review 
Principle  of  Response  Workload  (see  Appendix  A). 

(9)  Assessments  should  be  made  of  the  effects  of  HSI  upgrades  upon  the  crew’s  interactions  and  coordination,  as 
defined  in  fimction  and  task  analyses.  The  evaluations  should  ensure  that  the  HSI  design  supports  communication 
and  flexible  allocation  of  tasks  between  crew  members,  is  compatible  with  the  crew’s  organizational  structure,  and 
supports  manageable  workloads  while  ensuring  plant  safety.  These  tests  should  especially  consider 

•  Decision  aids  that  provide  special  information  to  an  individual  but  not  to  all  crew  members  (e.g.,  the  user 
may  experience  difficulty  communicating  complicated  concepts  to  the  rest  of  the  crew  because  they  do  not 
share  a  common  view  of  plant  conditions) 

•  Characteristics  of  display  systems  that  interfere  with  the  ability  of  personnel  to  share  a  view  of  the  plant  or 
to  maintain  awareness  of  other  operators’  actions 

•  Physical  characteristics  of  HSI  upgrades  that  may  impede  face-to-face  communications,  such  as 
components  that  interfere  with  the  line-of-sight  between  operators  or  equipment  that  requires  them  to 
spend  more  time  away  fi-om  the  main  control  panel. 

•  Other  physical  characteristics  of  the  HSI  that  may  impair  crew  coordination,  such  as  background  noise 
that  can  affect  the  intelligibility  of  speech. 

Discussion:  Industrial  experience  showed  that  introducing  computer-based  HSI  technologies  may  affect  crew 
coordination  in  complex  ways  (Hutchins,  1990;  Stubler  and  O’Hara,  1996b;  O’Hara,  Higgins,  Stubler,  and 
Kramer,  2000).  This  guideline  is  an  application  of  the  High-Level  Design  Review  Principle  of  Task  Compatibility 
(see  Appendix  A).  (For  additional  information  see  Crew  Performance  and  Team  Skills  in  Section  4.4.2.) 
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9.9  Procedure  Development 

Guidance  for  the  Applicability  of  the  NUREG-07!  1  Element 

(1)  For  all  upgrades  identified  in  Section  9.1,  a  review  of  procedure  development  should  be  conducted,  in  accordance 
with  Section  9  of  NUREG-071 1.  Procedure  development  should  address  all  personnel  tasks,  described  in  Section  9 
of  NUREG-07 1 1,  that  are  affected  by  the  upgrade  or  its  interactions  with  the  rest  of  the  plant. 

Discussion:  Because  procedures  are  an  essential  component  of  HSI  design,  they  should  be  kept  current  with  the 
design  of  the  plant  and  the  HSI.  This  guideline  extends  the  criteria  of  Section  9  ofNUREG-071 1  to  upgrades. 

(For  additional  information,  see  Section  5.3.2.) 

Guidance  Tailored from  NUREG-07 1 1  for  Upgrades 

(2)  Procedures  should  be  developed  or  modified  to  reflect  the  characteristics  and  functions  of  the  upgrade. 

Discussion:  This  guideline  is  an  extension  of  the  criteria  of  Section  9  of  NUREG-071 1  to  upgrades.  (For 
additional  information  see  Section  5.3.2.) 

(3)  Procedural  modifications  should  be  integrated  across  the  full  set  of  procedures;  alterations  in  particular  parts  of  the 
procedures  should  not  conflict  nor  be  inconsistent  with  other  parts. 

Discussion:  Conflicts  and  inconsistencies  may  confuse  operators  and  lower  the  usability  of  the  procedures.  This 
guideline  extends  the  criteria  of  Section  9  of  NUREG-071 1  to  upgrades.  (For  additional  information  see  Section 
5.3.2.) 

(4)  All  procedures  should  be  verified  as  denoted  in  Criterion  6  of  Section  9.4  of  NUREG-071 1  to  ensure  their  adequate 
content,  format,  and  integration.  The  procedures  also  should  be  assessed  through  validation  if  an  upgrade 
substantially  changes  personnel  tasks  that  are  significant  to  plant  safety.  The  validation  should  ensure  that  the 
procedures  correctly  reflect  the  characteristics  of  the  upgraded  plant  and  can  be  carried  out  effectively  to  restore  the 
plant. 

Discussion:  Procedures  should  be  evaluated  similarly  to  other  portions  of  the  HSI  because  they  are  an  essential 
component  of  its  design.  This  guideline  extends  to  upgrades  the  criteria  of  Section  9  of  NUREG-07 1 1 .  (For 
additional  information  see  Section  5.3.2.) 

New  Guidance  for  Upgrades  Only 

(5)  Procedures  should  be  developed  for  temporary  configurations  of  HSI  components  and  plant  systems  that  are  used 
by  operations  or  maintenance  personnel  when  the  plant  is  not  shut  down. 

Discussion:  Because  procedures  are  an  essential  component  of  HSI  design,  they  should  be  current  with  the  design 
of  the  plant  and  the  HSI.  This  guideline  is  an  extension  of  the  criteria  of  Section  9  of  NUREG-071 1  to  the  case  of 
upgrades.  (For  additional  information  see  Section  5.3.2.) 
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9.10  Training  Program  Development 

Guidance  for  the  Applicability  of  the  NUREG~0711  Element 

(1)  This  review  should  be  conducted,  in  accordance  with  Section  10  of  NUREG-071 1,  for  all  upgrades  identified  in 
Section  9.1.  Development  of  the  training  program  should  address  all  personnel  tasks,  described  in  Section  10, 
NUREG-071 1,  that  are  affected  by  the  upgrade  or  its  interactions  with  the  rest  of  the  plant. 

Discussion:  Training  is  an  important  factor  for  ensuring  the  plant’s  safe,  reliable  operation  (NUREG-071 1). 

New  Guidance  for  Upgrades  Only 

(2)  If  upgrades  are  to  be  implemented  over  time  so  that  temporary  configurations  of  plant  equipment  or  HSI 
components  will  be  created,  then  training  should  be  provided  for  each  configuration  that  personnel  will  use. 
Discussion:  Industrial  experience  has  shown  that  temporary  HSI  and  plant  configurations  due  to  upgrading  the 
plant  over  time  can  pose  demands  on  human  performance  that  differ  from  either  the  initial  or  final  configurations. 
(See  Section  5.4  for  additional  information.) 

(3)  Team  training  should  be  addressed  when  the  upgrade  significantly  affects  personnel’s  interaction  (e.g.,  increasing 
the  need  for  personnel  to  interact  or  changing  the  means  available  to  do  so),  and  if  these  interactions  are  important 
to  the  plant’s  safety.  This  training  should  cover  teamwork  skills  if  the  upgrade  requires  them.  Groups  of  people 
who  work  together  should  be  specifically  trained  in  team-related  tasks,  including  how  individual  roles  are  related, 
how  the  team’s  performance  depends  upon  individual  performances,  and  what  makes  the  team’s  task  different 
from  the  sum  of  the  individual  ones. 

Discussion:  The  development  of  team  skills  is  recognized  as  being  critical  to  safe,  effective  performance  in 
complex  human-machine  systems  (Mumaw  et  al.,  1994;  Swezey  and  Llaneras,  1997).  (For  additional  information 
see  Sections  5.3.3.5,  4.3,  and  4.4.) 

(4)  Operators’  training  for  upgrades  should  address  the  following  points: 

•  The  plant  goals  and  subgoals  affected  by  the  upgrade 

•  Rules  for  decision  making  relating  to  the  upgrade,  such  as  its  operation  and  recognition  of  faults 

•  Effective  strategies  for  accessing  and  processing  information  presented  in  the  upgraded  HSI  and  for  taking 
actions. 

This  information  should  be  presented  in  a  way  consistent  with  skills  that  personnel  already  possess.  They  should 
be  instructed  in  how  their  existing  skills  may  be  applied  to  the  upgraded  HSI. 

Discussion:  Goal  assessment,  effective  use  of  decision-making  rules,  and  strategies  for  gathering  and  processing 
information  are  important  elements  of  skilled  performance  (Mumaw  et  al.,  1994).  Instructing  personnel  in  using 
these  skills  can  take  them  to  higher  levels  of  performance  faster  than  if  they  must  develop  and  learn  these  skills  by 
themselves  (Mumaw  et  al.,  1994).  Industry  experience  shows  that  learning  and  job  performance  are  enh2inced 
when  personnel  can  transfer  their  skills  from  the  old  to  the  new  HSI  design  (Schlager  et  al.,  1989).  (For  additional 
information  see  Sections  5. 3. 3. 5, 4.3,  and  4.4.) 
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(5)  If  older  HSI  components  are  to  be  maintained  as  backups  to  newer  HSI  components  that  require  different  skills, 
then  the  training  program  should  ensure  that  personnel  can  maintain  adequate  levels  of  skill  for  both.  This 
training  should  also  provide  skills  for  transitioning  between  the  old  and  new  components. 

Discussion:  Skills  for  using  the  older  HSI  component  may  deteriorate  if  adequate  training  is  not  given  (Roth  and 
O’Hara,  1998).  (For  additional  information  see  Section  5.3.3.5,  Team  Training.) 

New  Guidance  for  the  NUREG-07J 1  Element 

(6)  Learning  objectives  for  personnel  training  should  address  the  knowledge  and  skill  requirements  associated  with  all 
relevant  dimensions  of  the  trainee’s  job,  such  as  interactions  with  the  plant,  the  HSI,  and  other  personnel.  Table 

9.1,  below,  shows  these  dimensions. 

Discussion:  Experiences  in  the  nuclear  power  industry  and  other  process-control  industries  have  indicated  that 
personnel  have  not  always  obtained  appropriate  training  for  new  technologies  because  the  analyses  of  training 
needs  were  incomplete  or  inadequate.  A  systematic  approach  to  identifying  training  needs  is  specified  in  Section 
10.4  of  NUREG-071 1,  10  CFR  55.4  ,  and  NUREG-0800.  (These  dimensions  are  described  further  in  Section 

5.3.3.1. ) 


Table  9.1  Some  Knowledge  and  Skill  Dimensions  for  Training-Needs  Assessment 


Topic 

Knowledge 

Skill 

Plant 

Interactions 

Understanding  of  plant  processes, 
systems,  operational  constraints, 
and  failure  modes 

Skills  associated  with  monitoring 
and  detection,  situation  aweireness, 
response  planning  and 
implementation 

HSI 

Interactions 

Understanding  of  HSI  structure, 
functions,  failure  modes,  and 
interface  management  tasks 
(actions,  errors,  and  recovery 
strategies) 

Skills  associated  with  interface- 
management  tasks 

Personnel 

Interactions 

(In  the  CR  and  in  the  plant) 

Understanding  information 
requirements  of  others,  how 
actions  must  be  coordinated  with 
others,  policies  and  constraints  on 
crews’  interaction 

Skills  associated  with  crew’s 
interactions  (i.e.,  teamwork) 

(7)  Factual  knowledge  should  be  taught  within  the  context  of  actual  tasks  so  that  personnel  learn  to  apply  it  in  the 
work  environment.  The  context  of  the  job  should  be  defined,  and  it  should  be  represented  meaningfully  to  help 
trainees  to  link  the  knowledge  to  the  job’s  requirements.  Training  that  addresses  theory  should  be  integrated  with 
training  in  the  use  of  procedures. 

Discussion:  Inert  knowledge  (Mumaw  et  al.  1994;  Mann  and  Hammer,  1986)  is  knowledge  that  can  be  applied  in 
a  training  setting  but  cannot  be  used  effectively  in  the  actual  task  setting.  It  fails  to  become  part  of  the  usable  store 
of  the  trainee’s  knowledge  because  it  is  not  tied  into  the  context  of  task  performance.  (For  additional  information, 
see  Section  5.3.3.2,  Approaches  to  Training  Cognitive  Skills.) 
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(8)  Training  should  address  the  relationships  between  the  plant’s  goals  and  subgoals  affected  by  the  upgrade.  It  should 
support  personnel  in  determining  the  goals  most  relevant  to  the  current  state  of  the  plant,  assessing  possible  effects 
that  alternative  paths  may  have  on  other  goals,  selecting  paths  for  achieving  the  most  relevant  goals,  and  ensuring 
that  actions  specified  by  procedures  are  consistent  with  those  goals  and  paths. 

Discussion:  Goal  assessment  is  an  important  element  of  skilled  problem  solving  and  decision  making  (Mumaw  et 
al.,  1994).  (For  additional  information,  see  Section  5.3.3.2,  Approaches  to  Training  Cognitive  Skills,  and  Section 
5. 3. 3.3,  Training  for  Conceptual  Knowledge.) 

(9)  Training  should  address  rules  for  decision  making  related  to  plant  systems  and  the  HSI.  It  should  include  rules  for 
accessing  and  interpreting  information  provided  by  HSI  components,  and  rules  for  interpreting  symptoms  of 
failures  of  systems  or  the  HSI.  This  training  should  cover  acquiring  new  decision-making  rules  and  eliminating 
existing  ones  that  are  not  appropriate  to  the  plant  and  HSI  design. 

Discussion:  Decision-making  rules  are  an  important  aspect  of  skilled  performance.  Instructing  personnel  using 
correct  and  efficient  rules  can  bring  them  to  high  levels  of  performance  more  quickly  than  if  they  themselves  must 
develop  and  learn  the  rules  (Mumaw  et  al.,  1994).  (For  additional  information,  see  Section  5. 3. 3. 2,  Approaches  to 
Training  Cognitive  Skills,  and  Section  5.3.3.3,  Training  for  Conceptual  Knowledge.) 

(10)  Operators  should  be  trained  in  the  use  of  effective  strategies  for  accessing  and  processing  information  from  the 
upgraded  HSI.  This  training  should  support  personnel  in  acquiring  new  strategies,  modifying  existing  ones,  and 
eliminating  inappropriate  strategies. 

Discussion:  Strategies  for  gathering  and  processing  information  are  important  for  achieving  high  levels  of  skilled 
performance.  Instructing  personnel  in  using  them  correctly  and  efficiently  can  enable  personnel  to  achieve  high 
levels  of  performance  more  quickly  than  if  they  must  develop  and  learn  them  by  themselves  (Mumaw  et  al.,  1994). 
(For  additional  information  see  Section  5.3.3.2,  Approaches  to  Training  Cognitive  Skills.) 

(11)  Training  programs  for  developing  skills  should  be  structured  so  that  the  training  environment  is  consistent  with 
the  level  of  skill  being  taught.  It  should  support  skill  acquisition  by  allowing  trainees  to  manage  cognitive 
demands.  For  example,  trainees  should  not  be  placed  in  environments  teaching  high-level  skills,  such  as 
coordinating  control  actions  among  crew  members,  before  they  have  mastered  requisite,  low-level  skills,  such  as 
how  to  manipulate  control  devices. 

Discussion:  The  effective  management  of  cognitive  resources  can  support  the  acquisition  of  skills,  making 
training  more  effective  and  enhancing  performance  in  the  work  environment  (Mumaw  et  al.,  1994).  (For  further 
information,  see  Section  5.3.3.4,  Training  for  Skill  Automaticity  and  Section  5.3.3.6,  Evaluation  of  Training 
Systems .) 

9.11  Human  Factors  Verification  and  Validation 

9.11.1  General  Criteria 

Guidance  for  the  Applicability  of  the  NUREG-071I  Element 

(1)  Verification  and  validation  should  be  conducted  in  accordance  with  the  general  criteria  in  Section  1 1.4.1  of 

NUREG-071 1.  The  general  scope  of  the  V&V  activities  should  include  all  items,  defined  in  Criterion  1  of  Section 
1 1 .4. 1  of  NUREG-07 1 1 ,  that  are  applicable  to  the  upgrade.  Plans  for  the  V&V  and  reports  of  the  results  should  be 
documented  as  described  in  Section  1 1 .3  of  NUREG-071 1 .  The  applicability  of  individual  verification  and 
validation  evaluations  is  described  in  the  three  subsections,  below. 
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Discussion:  This  guideline  was  derived  from  Section  1 1.4.1ofNUREG-071 1. 

9.1 1.2  HSI  Task  Support  Verification 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(2)  A  review  of  HSI  task  support  should  be  conducted,  in  accordance  with  Section  1 1 .4.2  NUREG-07 1 1 ,  for  all 
upgrades  identified  in  Section  9.1.  The  HSI  verification  should  address  all  HSI  aspects  described  in  Section  1 1.4.2 
of  NUREG-07 1 1  that  are  relevant  to  the  upgrade.  For  upgrades  that  do  not  include  modifications  to  the  HSI,  task 
support  verification  should  identify  any  new  demands  for  monitoring  and  control,  and  determine  whether  they  are 
adequately  addressed  by  the  existing  HSI  design. 

Discussion:  System  upgrades  sometimes  impose  new  monitoring  and  control  demands  on  personnel.  An  HSI  task 
support  verification  can  identify  and  evaluate  these  demands.  (For  additional  information,  see  Sections  5.4.1  and 
5.4.4.) 

New  Guidance  for  Upgrades  Only 

(3)  Task  support  verification  should  address  upgrade  configurations  in  which  old  HSI  components  are  permanently 
deactivated,  but  not  removed  (e.g.,  abandoned  in  place).  Criterion  2  of  Section  1 1.4.2  states  that  the  HSI  should 
not  contain  any  information,  displays,  or  controls  that  do  not  support  the  operator’s  tasks.  This  verification  should 
identify  deactivated  HSI  components  that  may  have  potentially  negative  effects  on  personnel’s  performance,  such 
as  obstructing  the  view  of  important  information  or  adding  visual  clutter  which  may  interfere  with  monitoring. 
Deactivated  HSI  components  requiring  further  evaluation  through  HFE  design  verification  or  integrated  system 
validation  should  be  identified. 

Discussion:  For  additional  information,  see  Sections  5.4.1  and  5.4.4. 

(4)  Task  support  verification  should  address  temporary  configurations  of  the  HSI  and  plant  systems  that  may  be 
created  during  implementation  of  the  upgrade,  and  used  by  operations  and  maintenance  personnel  when  the  plant 
is  not  shut  down.  These  configurations  may  include: 

•  The  use  of  HSI  components  that  differ  from  the  intended  final  design 

•  Combinations  of  HSI  components  and  system  configurations  that  differ  from  both  the  original  and  the 
intended  final  designs. 

For  each  temporary  HSI  configuration,  task  requirements  should  be  identified  and  compared  to  the  information  and 
control  capabilities  provided.  For  example,  if  a  temporary  configuration  of  plant  systems  introduces  special 
monitoring  requirements,  then  the  HSI  should  provide  the  necessary  information. 

Discussion:  The  operation  and  maintenance  of  temporary  configurations  of  plant  systems  or  HSI  components  can 
impose  unique  demands  on  human  capabilities.  (For  additional  information,  see  Sections  5.4.1  and  5.4.4.) 
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9.11.3  HFE  Design  Verification 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(5)  HFE  design  verification  should  be  reviewed,  in  accordance  with  Section  1 1 .4.3  of  NUREG-071 1,  for  all  upgrades 
of  the  HSI  identified  in  Section  9.1.  Its  scope  may  be  restricted  to  the  upgraded  HSI  components  and  their 
interactions  with  the  rest  of  the  HSI. 

Discussion:  HFE  design  verification  is  only  performed  on  the  HSI,  not  on  upgrades  of  plant  systems.  (NUREG- 
071 1). 

New  Guidance  for  Upgrades  Only 

(6)  When  both  old  and  new  versions  of  similar  HSI  components  are  permanently  present  in  the  HSI,  this  verification 
should  ensure  that  their  means  of  presentation  and  methods  of  operation  are  compatible,  such  that  personnel 
performance  will  not  be  impaired  when  using  them  alternately. 

Discussion:  Incompatibilities  may  result  in  decrements  in  human  performance  when  their  use  is  alternated.  (For 
additional  information,  see  Sections  5.4.2  and  5.4.4.) 

(7)  Temporary  configurations  of  the  HSI  and  plant  systems,  which  may  be  used  by  operations  and  maintenance 
personnel  when  the  plant  is  not  shut  down,  should  be  reviewed  to  ensure  that  their  design  is  consistent  with  the 
principles  of  good  HFE  design,  including  consistency  with  the  rest  of  the  HSI. 

Discussion:  Industry  experience  indicates  that  when  the  HSI  or  plant  systems  are  upgraded,  plant  personnel  may 
encounter  temporary  configurations  which  they  must  operate  and  maintain.  These  may  impose  demands  that  differ 
from  the  initial  and  final  configurations.  (For  additional  information  see  Sections  5.4.2  and  5.4.4.) 

9.11.4  Integrated  System  Validation 

Guidance  for  the  Applicability  of  the  NUREG-0711  Element 

(8)  The  integrated  system  validation  should  be  reviewed,  in  accordance  with  Section  1 1 .4.4  NUREG-07 1 1 ,  for  all 
upgrades  identified  in  Section  9.1  that  may  (1)  change  personnel’s  tasks;  (2)  change  task  demands,  such  as 
changing  a  task’s  dynamics,  complexity,  or  workload;  or  (3)  interact  with  or  affect  other  HSI  components  in  ways 
that  may  degrade  performance.  Integrated  system  validation  may  not  be  needed  when  an  upgrade  results  in  minor 
changes  to  personnel  tasks  such  that  they  may  reasonably  be  expected  to  have  little  or  no  overall  effect  on  workload 
and  the  likelihood  of  error.  The  scope  of  the  integrated  system  validation  should  include  the  upgrade  and  its 
interactions  with  the  rest  of  the  plant. 

Discussion:  Integrated  system  validation  is  the  process  by  which  an  integrated  system  design  (i.e.,  hardware, 
software,  and  personnel  elements)  is  evaluated  to  determine  whether  it  acceptably  supports  safe  operation  of  the 
plant.  It  is  intended  to  evaluate  the  acceptability  of  those  aspects  of  the  design  that  cannot  be  determined  though 
such  analytical  means  as  HSI  task  support  verification  and  HFE  design  verification.  For  upgrades  that  change 
plant  systems  but  do  not  modify  the  HSI,  validation  can  provide  evidence  about  the  adequacy  of  the  existing  HSI 
for  supporting  personnel  performance.  (For  additional  information  see  Sections  5.4.3  and  5.4.4.) 
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9.11.4.1  Test  Objectives 

Guidance  Tailored  from  NUREG-071 1  for  Upgrades 

(9)  Section  1 1. 4.4.2  of  NUREG-071 1  states  that  the  detailed  test  objectives  should  identify  aspects  of  the  integrated 
system,  including  staffing,  communications,  and  training,  that  may  negatively  affect  integrated  system 
performance.  This  scope  may  be  limited  to  HSI  components  included  in  the  upgrade,  other  HSI  components  used 
with  the  upgrade,  and  the  potential  interactions  between  the  two. 

Discussion:  Integrated  system  validation  should  address  aspects  of  integrated  system  performance  that  are  related 
to  the  upgrade,  including  interactions  between  HSI  components.  For  example,  a  new  CRT-based  display  system 
may  be  negatively  affected  by  the  existing  lighting  system  of  the  CR.  A  new  HSI  component  that  emits  high  levels 
of  background  noise  may  degrade  the  effectiveness  of  the  existing  audible  alarm  system  in  the  CR.  (For  additional 
information  see  Sections  5.4.3  and  5.4.4.) 

(10)  The  test  objectives  and  scenarios  should  be  developed  to  exercise  dimensions  of  personnel  performance  that  are 
addressed  by  the  design  of  the  upgrade,  including  functions  and  tasks  affected  by  the  upgrade. 

Discussion:  Integrated  system  validation  should  focus  on  asjjects  of  integrated  system  performance  related  to  the 
upgrade.  This  guideline  extends  the  criteria  of  Section  1 1 .4.4  of  NUREG-071 1 .  (For  additional  information,  see 
Sections  5.4.3  and  5.4.4.) 

9.11.4.2  Validation  Testbeds 

Guidance  Tailored from  NUREG-07 1 J  for  Upgrades 

(11)  The  validation  testbeds  should  include  procedures  that  have  been  modified  for  the  upgrades. 

Discussion:  Procedures  are  an  important  component  of  the  HSI  (NUREG-071 1). 

9.11.4.2.1  Main  Control  Room 

Guidance  Tailored  from  NUREG-0711  for  Upgrades 

(12)  If  a  training  simulator  is  used  as  the  validation  testbed  for  the  main  control  room,  then 

•  The  user  interfaces  of  the  training  simulator  should  be  modified  to  accurately  represent  characteristics  of 
the  HSI  upgrade. 

•  The  simulation  of  the  plant’s  response  should  be  modified  to  reflect  the  behavior  of  the  upgrade. 
Discussion:  This  guideline  is  an  extension  to  upgrades  of  the  guidance  of  Section  1 1.4.4.3.1  of  NUREG-071 1. 

(13)  If  a  training  simulator  is  not  used  as  the  validation  testbed  for  the  main  control  room,  then  other  suitable  testbeds 
should  be  developed,  such  as  part-task  simulators  and  mockups.  These  testbeds  should 

•  Represent  the  HSI  components  of  the  upgrade 

•  Represent  interactions  with  HSI  components  and  plant  systems  that  are  related  to  the  upgrade 
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•  Represent  aspects  of  the  task  environment  that  are  relevant  to  the  human  performance  considerations 
identified  early  in  the  HFE  design  review 

•  Be  consistent  with  the  dimensions  of  testbed  fidelity  described  in  Section  1 1.4.4.3.1of  NUREG-071 1. 
Discussion:  This  guideline  extends  the  guidance  of  Section  1 1. 4.4.3. 1  of  NUREG-071 1  to  upgrades. 

9.11.4.2.2  Representation  of  Facilities  Remote  from  the  Main  Control  Room 
New  Guidance  for  the  NUREG-0711  Element 

(14)  When  simulations  or  mockups  are  used,  the  testbed  should  represent  all  aspects  of  the  environment  that  are 
relevant  to  the  human  performance  factors  identified  in  the  earlier  stages  of  the  HFE  design  review,  including  the 
following: 

•  Static  and  dynamic  characteristics  of  the  HSI,  including  manual  and  automatic  operating  features  (e.g., 
automatic  mode  changes)  and  faults 

•  Static  and  dynamic  characteristics  of  the  plant’s  systems,  including  their  behavior  and  fault  indications 

•  Communications  with  personnel  located  in  the  control  room  or  elsewhere  in  the  plant 

•  Task-environment  factors  that  may  affect  personnel’s  performance  (lighting,  noise,  heating  and 
ventilation,  and  protective  clothing  and  equipment). 

Discussion:  This  guideline  is  a  revision  Criterion  2  of  Section  1 1 .4 .4.3.2  of  NUREG-071 1 .  Operational  and 
maintenance  tasks  performed  at  local  control  stations  and  local  maintenance  interfaces  can  affect  the  plant’s  safety 
(Stubler,  Higgins,  and  Kramer,  2000). 

9.11.4.3  Plant  Personnel 

New  Guidance  for  the  NUREG-0711  Element 

(15)  The  selection  of  participants  for  particular  validation  trials  should  be  consistent  with  the  test’s  objectives.  For 
trials  requiring  CR  operating  shifts,  consideration  should  be  given  to  the  assembly  of  operating  crews  (e.g.,  shift 
supervisors,  reactor  operators,  and  shift  technical  advisors)  that  will  participate  in  the  tests.  For  trials  that  address 
maintenance  tasks,  maintenance  personnel  should  be  selected  with  expertise  in  the  appropriate  area  (e.g., 
mechanical,  electrical,  or  instrumentation  and  control. 

Discussion:  This  guideline  is  a  revision  of  Criterion  3  of  Section  1 1 .4.4.4  of  NUREG-071 1 .  It  was  expanded  to 
include  some  considerations  related  to  upgrades  and  new  plant  designs.  Integrated  system  validation  should 
address  risk-important  maintenance  tasks,  such  as  those  undertaken  while  the  plant  is  on-line.  (For  additional 
information,  see  Sections  5.4.3  and  5.4.4.) 
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9.11.4.4  Operational  Conditions 

Guidance  Tailored from  NUREG-0711  for  Upgrades 

(16)  The  personnel  tasks  used  during  the  validation  trials  should  reflect  tasks  and  HSI  aspects  that  involve  the  upgrade, 
rather  than  the  entire  range  of  topics  discussed  in  Criterion  3  of  Section  1 1 .4.4.5. 

Discussion:  The  guideline  extends  the  criteria  of  Section  1 1. 4.4.5  ofNUREG-071 1  to  validating  upgrades. 

(17)  The  operational  conditions  used  during  the  validation  trials  should  address  the  ability  of  personnel  to  comprehend 
the  limits  of  integration  between  components  of  the  upgrade,  and  between  the  upgrade  and  the  rest  of  the  plant. 

For  example,  operators  should  be  able  to  determine  whether  similar  plant  variables,  presented  via  different  display 
devices  or  display  pages,  are  identical,  or  whether  their  data  originate  from  different  sources. 

Discussion:  Industrial  experience  suggests  that  the  displays  of  computer-based  systems  can  obscure  boundaries 
between  functionally  separate  systems  or  suggest  levels  of  integration  that  do  not  exist.  (See  Sections  5.3.1,  5.4.3 
and  5.4.4  for  additional  information.) 

New  Guidance  for  Upgrades  Only 

(18)  The  operational  conditions  used  during  the  validation  trials  should  address  the  effects  of  transfer  of  learning  effects 
on  personnel’s  performance  when  an  upgrade  replaces  an  older  HSI  component.  These  trials  should  determine 
whether  the  training,  procedures,  and  HSI  design  are  sufficient  to  prevent  negative  transfer  of  learning  which  may 
affect  plant  safety. 

Discussion:  Negative  transfer  of  learning  effects  may  occur  when  the  new  and  old  components  are  different  and 
impose  different  demands  on  personnel  using  them.  (See  Sections  5.3.1, 5.4.3  and  5.4.4  for  additional 
information.) 

(19)  When  both  old  and  new  versions  of  the  same  HSI  components  with  different  means  of  presentation  and  methods  of 
operation  are  permanently  present  in  the  HSI,  integrated  validation  should  ensure  that  personnel  can  alternate  their 
use  of  these  HSI  components  without  degrading  their  performance. 

Discussion:  Incompatibilities  between  the  methods  of  operating  HSI  components  may  cause  decrements  in  human 
performance  when  their  use  is  alternated.  (See  Sections  5.4.3  and  5.4.4  for  additional  information.) 

(20)  If  HSI  task  support  verification  identifies  potential  human  factors  discrepancies  associated  with  old  HSI 
components  that  are  to  be  deactivated  and  left  in  place  in  the  HSI,  then  validation  tests  should  be  developed  for 
these  problems. 

Discussion:  The  presence  of  deactivated  HSI  components  may  have  negative  effects  on  personnel,  such  as  causing 
visual  clutter  that  interferes  with  people’s  ability  to  locate  and  use  other  HSI  components.  (See  Sections  5.4.3  and 
5.4.4  for  additional  information.) 
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The  design  of  human-system  interfaces  (HSIs)  should  support  the  operating  personnel’s  primary  task  of  monitoring  and 
controlling  the  plant,  without  imposing  an  excessive  workload  associated  with  using  them  (manipulating  windows, 
selecting  display  selection,  and  navigating,  for  example).  The  HSI  also  should  support  the  recognition,  tolerance,  and 
recovery  from  human  errors.  Guidelines  for  reviewing  human  factors  engineering  design  help  to  ensure  that  these  goals  are 
achieved.  As  part  of  the  guidance  development  for  NUREG-0700,  Rev.  1,  a  set  of  “high-level”  design  review  principles 
were  established  representing  the  generic  HSI  characteristics  necessary  to  support  personnel’s  performance.  They  were 
used  to  draft  many  detailed  review  guidelines  in  Part  2  of  NUREG-0700  (see  O’Hara,  Brown,  and  Nasta,  1996  for  a 
discussion  of  their  use).  The  high-level  principles  also  were  used  in  the  formulating  guidelines  for  computer-based 
procedures. 

The  1 8  principles  are  divided  into  four  categories:  general  principles,  primary  task  design,  secondary  task  control,  and  task 
support .  The  categories  and  the  principles  that  comprise  them  are  described  below. 

General  Principles 

These  principles  ensure  that  the  HSI  design  supports  personnel  safety,  and  is  compatible  with  people’s  general  cognitive 
and  physiological  capabilities. 

•  Personnel  Safety  -  The  design  should  minimize  the  potential  for  injury  and  exposure  to  harmful  materials. 

•  Cognitive  Compatibility  -  The  operators’  role  should  consist  of  purposeful,  meaningful  tasks  that  enable  them  to 
remain  familiar  with  the  plant,  and  maintain  a  level  of  workload  that  is  not  so  high  as  to  negatively  affect 
performance,  but  sufficient  to  maintain  vigilance. 

•  Physiological  Compatibility  -  The  design  of  the  interface  should  reflect  physiological  characteristics,  including 
visual  and  auditory  perception,  biomechanics  (reach  and  motion),  motor  control,  and  anthropometry. 

•  Simplicity  of  Design  -  The  HSI  should  represent  the  simplest  design  consistent  with  functional  and  task 
requirements. 

•  Consistency  -  There  should  be  a  high  degree  of  consistency  between  the  HSI,  the  procedures,  and  the  training 
systems.  At  the  HSI,  the  way  the  system  functions  and  appears  to  the  operating  crew  always  should  reflect  a  high 
degree  of  standardization,  and  consistency  with  procedures  and  training. 

Primary  Task  Design 

These  principles  support  the  operator’s  primary  task  of  process  monitoring,  decision  making,  and  control  to  maintain  safe 
operation. 

•  Situation  Awareness  -  The  information  presented  to  the  users  by  the  HSI  should  be  correct,  rapidly  recognized,  and 
easily  understood  (e.g.,  “direct  perception”  or  “status-at-a-glance”  displays)  and  support  the  higher-level  goal  of 
user’s  awareness  of  the  system’s  status. 

•  Task  Compatibility  -  The  system  should  meet  the  requirements  of  users  to  perform  their  tasks  (including  operation, 
safe  shutdown,  inspection,  maintenance,  and  repair).  The  forms  and  formats  of  data  should  be  appropriate  to  the 
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task  (including  the  need  to  access  confirmatory  data  or  raw  data  in  the  case  of  higher-level  displays),  and  control 
options  should  encompass  the  range  of  potential  actions.  No  unnecessary  information  or  control  options  should  be 
present. 

•  User  Model  Compatibility  -  All  aspects  of  the  system  should  be  consistent  with  the  users’  mental  models 
(understanding  and  expectations  about  how  the  system  behaves,  learned  through  training,  using  procedures,  and 
experience).  All  aspects  of  the  system  also  should  be  consistent  with  established  conventions  (i.e.,  expressed  in 
customary,  commonplace,  useful,  and  functional  terms,  rather  than  abstract,  unusual  or  arbitrary  forms,  or  in  forms 
requiring  interpretation). 

•  Organization  of  HSI  Elements  -  The  organization  of  all  aspects  of  the  HSl  (from  the  elements  in  individual 
displays,  to  individual  workstations,  to  the  entire  control  room)  should  be  based  on  the  user’s  requirements  and 
should  reflect  the  general  principles  of  organization  by  importance,  frequency,  and  order  of  use.  Critical  safety 
function  information  should  be  available  to  the  entire  operating  crew  in  dedicated  locations  to  ensure  its 
recognition,  and  to  minimize  data  search  and  response. 

•  Logical/Explicit  Structure  -  All  aspects  of  the  system  (formats,  terminology,  sequencing,  grouping,  and  the 
operator’s  decision-support  aids)  should  reflect  an  obvious  logic  based  on  task  requirements  or  some  other  non- 
arbitrary  rationale.  There  should  be  a  clear  relationship  of  each  display,  control,  and  data-processing  aid  to  the 
overall  task  or  function.  The  structure  of  the  interface  and  its  associated  navigation  aids  should  make  it  easy  for 
users  to  recognize  where  they  are  in  the  data  space,  and  enable  them  to  rapidly  access  data  not  currently  visible 
(e.g.,  on  other  display  pages).  The  way  the  system  works  and  is  structured  should  be  clear  to  the  user. 

•  Timeliness  -  The  system’s  design  should  take  into  account  users’  cognitive  processing  capabilities  as  well  as 
process-related  time  constraints  to  ensure  that  tasks  can  be  performed  within  the  time  required.  Information-flow 
rates  and  control-performance  requirements  that  are  too  fast  or  too  slow  could  diminish  performance. 

•  Controls/Displays  Compatibility  -  Displays  should  be  compatible  with  the  requirements  for  data  entry  and  control. 

•  Feedback  -  The  system  should  provide  useful  information  on  its  status,  permissible  operations,  errors  and  error 
recovery,  dangerous  operations,  and  validity  of  data. 

Secondary  Task  Control 


These  principles  minimize  secondary  tasks,  i.e.,  tasks  personnel  must  perform  when  working  with  the  system  that  are  not 
directed  to  the  primary  task.  Secondary  tasks  include  managing  the  interface,  such  as  navigating  through  displays, 
manipulating  windows,  and  accessing  data.  Performing  secondary  tasks  detracts  from  the  crew’s  primary  tasks,  so  their 
demands  must  be  controlled. 

•  Cognitive  Workload-  The  information  presented  by  the  system  should  be  rapidly  recognized  and  understood; 

therefore,  its  design  should  minimize  the  need  for  mental  calculations  or  transformations,  and  use  of  recall 
memory  (recalling  lengthy  lists  of  codes,  complex  command  strings,  information  from  one  display  to  another,  or 
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lengthy  action  sequences).  Raw  data  should  be  processed  into  a  directly  usable  form  (although  raw  data  still 
should  be  accessible  for  confirmation). 

•  Response  Workload-  The  system  should  require  a  minimum  number  of  steps  to  accomplish  an  action,  e.g.,  single- 
versus  command-keying,  menu  selection  versus  multiple-command  entry,  single  input  mode  (keyboard,  mouse) 
versus  mixed  mode.  In  addition,  it  should  not  require  the  entry  of  redundant  data,  nor  the  re-entry  of  information 
already  present,  or  information  the  system  can  generate  from  data  already  resident. 

Task  Support 

These  principles  address  the  characteristics  of  the  HSI  that  support  its  use  by  personnel,  such  as  providing  (1)  HSI 

flexibility  so  tasks  can  be  accomplished  in  more  than  one  way,  (2)  guidance  for  users,  and  (3)  mitigation  of  errors. 

•  Flexibility  -  The  system  should  give  the  user  multiple  means  to  carry  out  actions  (and  verify  automatic  actions), 
and  permit  displays  and  controls  to  be  configured  in  the  most  convenient  way.  However,  flexibility  should  be 
limited  to  situations  where  it  is  advantageous  in  task  performance  (e.g.,  in  accommodating  different  levels  of 
experience  of  the  users);  flexibility  should  not  be  provided  for  its  own  sake  because  there  is  a  tradeoff  between 
consistency  and  the  imposition  of  interface  management  workload  (which  detracts  from  monitoring  and 
operations). 


User  Guidance  and  Support -T\it  system  should  provide  an  effective  “help”  function;  i.e.,  informative,  easy-to- 
use,  and  relevant  guidance  should  be  provided  on-line  and  off-line  to  help  the  user  understand  and  operate  the 
system. 

Error  Tolerance  and  Control- A  fail-safe  design  should  be  provided  wherever  failure  can  damage  equipment, 
injure  personnel,  or  inadvertently  operate  critical  equipment.  Therefore,  the  system  should  generally  be  designed 
so  that  a  user’s  error  will  not  have  serious  consequences.  The  negative  effects  of  errors  should  be  controlled  and 
minimized.  The  system  should  offer  simple,  comprehensible  notification  of  the  error,  and  simple,  effective 
methods  for  recovery. 
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Abilities:  Usually  cognitive  capabilities  that  require  applying  some  knowledge  base,  and  are  necessary  to  perform  a  job 
function. 

Automaticity:  The  ability  to  perform  specific  sets  of  actions  without  committing  extensive  cognitive  resources. 

Capture  error:  An  error  of  execution  (slip)  that  occurs  when  an  infrequent  action  requires  a  sequence  of  operations,  some 
of  which  are  the  same  as,  or  similar  to,  those  of  a  frequent  action.  In  attempting  the  infrequent  action,  the  more  frequent 
one  is  performed  instead.  For  example,  an  operator  intends  to  perform  task  1,  which  is  composed  of  operations  A,  B,  C, 
and  D,  but  instead  executes  the  more-frequently  performed  task  2,  composed  of  operations  A,  B,  C,  and  E. 

Data-driven  monitoring:  Monitoring  that  depends  on  the  relative  and  absolute  levels  of  salience.  The  strength  of  the 
stimuli  must  exceed  some  threshold  to  draw  attention,  above  which  personnel  tend  to  attend  to  the  stimuli  with  the  greatest 
salience.  (Contrasts  with  model-driven  monitoring.) 

Description  error:  An  error  of  execution  (slip)  that  involves  performing  the  wrong  set  of  well-practiced  actions. 
Description  errors  occur  when  the  information  that  activates  or  triggers  the  action  is  either  ambiguous  or  undetected. 

Detection:  Recognition  that  something  is  not  operating  correctly,  and  that  an  abnormality  exists. 

Emergent  feature:  A  "high-level,"  global  perceptual  feature  produced  by  the  interactions  among  individual  parts  or 
graphical  elements  of  a  display  (e.g.,  lines,  contours,  and  shapes). 

Knowledge:  An  organized  body  of  information,  usually  factual  or  procedural,  the  application  of  which  allows  adequate  job 
performance.  It  is  the  foundation  upon  which  abilities  and  skills  are  built. 

Long-term  memory:  A  mental  repository  of  things  that  have  been  learned.  Its  capacity  for  storing  information  is  large 
(i.e.,  considered  by  some  to  be  virtually  unlimited). 

Loss-of-activation  error:  An  error  (slip)  that  occurs  when  an  intended  action  is  not  carried  out  due  to  a  failure  of  memory 
(i.e.,  the  intention  has  partially  or  completely  decayed  from  memory).  A  special  case  of  loss-of-activation  errors  involves 
forgetting  part  of  an  intended  act  while  remembering  the  rest  (e.g.,  retrieving  a  display  while  not  being  able  to  remember 
why  it  is  needed). 

Mental  model:  The  operator’s  internal  representation  of  physical  and  functional  characteristics  of  the  plant  and  its 
operation. 

Misordered  components  of  an  action  sequence:  An  error  (slip)  involving  skipped,  reversed,  or  repeated  steps. 

Mistake:  An  error  in  intention  formation,  such  as  forming  a  plan  inappropriate  for  the  situation.  Mistakes  are  related  to 
incorrect  assessments  of  the  situation,  or  inadequate  planning  of  a  response.  (Contrasts  with  slip.) 
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Mode  error:  An  operation  that  is  appropriate  for  one  mode  when  the  device  is  in  another  mode.  Mode  errors  occur  when 
the  user  believes  the  device  is  in  one  mode  when  it  is  in  another,  and  consequently,  performs  an  action  that  is  inappropriate 
for  the  actual  mode. 

Model-driven  monitoring:  Monitoring  that  is  directed  by  the  individual’s  knowledge  and  expectations.  It  can  be  viewed 
as  active  monitoring  in  the  sense  that  personnel  deliberately  direct  attention  to  areas  they  expect  to  provide  useful 
information,  rather  than  merely  attending  to  stimuli  based  on  their  relative  salience.  (Contrasts  with  data-driven 
monitoring.) 

Monitoring:  Checking  the  state  of  the  plant  to  determine  whether  the  systems  and  equipment  are  operating  correctly.  This 
may  include  checking  parameters  indicated  on  the  control  room  panels,  monitoring  parameters  displayed  by  the  process 
computer,  and  obtaining  verbal  reports  from  personnel. 

Primary  tasks:  Tasks  performed  by  operators  as  part  of  their  functional  role  of  supervising  the  plant.  Primary  tasks 
require  knowledge  of  the  plant  and  cognitive  skills  for  making  decisions  and  taking  actions. 

Response  implementation:  Performing  actions  identified  during  response  planning. 

Response  planning:  Developing  an  approach  for  achieving  a  goal.  The  planned  response  may  be  as  simple  as  deciding  to 
access  a  particular  parameter  from  a  particular  human-system  interface  component,  or  it  may  involve  selecting  a 
complicated  course  of  action  guided  by  a  procedure. 

Secondary  tasks:  Tasks  that  operators  must  perform  but  which  are  not  directly  related  to  a  primary  task.  Secondary  tasks 
involving  management  of  the  human-system  interface  include  navigating  through  an  information  system,  manipulating 
windows  on  a  display,  and  manipulating  performance  aids. 

Selective  attention:  The  ability  to  shift  attention  to  important  things  and  ignore  irrelevant  ones.  Selective  attention  is 
driven  by  both  bottom-up  and  top-down  processes.  The  former  are  based  on  characteristics  of  the  stimuli  that  affect 
salience,  such  as  their  size,  loudness,  and  brightness,  and  results  in  attention  to  those  with  highly  salient  features  (see  data- 
driven  monitoring).  Top-down  processes  are  developed  from  past  experiences  and  are  based  on  a  person’s  understanding  of 
the  environment  (i.e.,  mental  model)  and  expectations.  Top-down  processing  results  in  attention  to  stimuli  that  have 
special  significance  (i.e.,  see  model-driven  monitoring). 

Situation  awareness:  The  cognitive  activities  involved  in  processing  stimuli  to  construct  an  understanding  of  the  situation. 
Two  important  ones  are  interpreting  the  current  state  of  the  plant,  and  determining  its  implications. 

Situation  model:  An  understanding  of  the  specific  current  situation  (i.e.,  the  current  state  of  the  plant),  based  on  factors 
known  or  hypothesized  to  be  affecting  the  situation  at  a  given  time. 

Slip:  An  error  in  executing  an  intention  (i.e.,  the  user  intends  to  do  one  thing  but  does  another).  (Contrasts  with  mistake.) 
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Skills:  The  capability  to  perform  a  job  with  ease  and  precision.  The  term  is  often  applied  to  both  psychomotor  and 
cognitive  activities.  For  example,  a  psychomotor  skill  may  entail  precisely  operating  a  particular  control  device.  A 
cognitive  skill  may  entail  using  strategies  for  rapidly  finding  and  accessing  information. 

Sustained  attention:  The  ability  to  attend  to  a  single  information  source  for  an  extended  period.  It  is  also  called  focused 
attention. 

Team:  A  set  of  two  or  more  people  who  interact  dynamically,  interdependently,  and  adaptively  toward  a  common  valued 
goal,  objective,  or  mission;  each  having  specific  roles  or  functions  to  perform. 

Transfer  of  training:  A  framework  for  describing  the  effects  of  prior  learning  upon  performance  in  a  new  (transfer) 
condition.  Positive  and  negative  transfer,  respectively,  refer  to  the  facilitative  and  inhibitory  effects  of  earlier  learning. 

Unintentional  activation:  A  slip  that  occurs  when  a  set  of  actions  that  is  not  part  of  a  current  action  sequence  becomes 
activated  for  extraneous  reasons,  and  then  becomes  triggered;  this  can  lead  to  the  unintended  actuation  of  an  input  device. 

Working  memory:  A  mental  repository  containing  things  that  are  currently  in  consciousness  (i.e.,  things  that  one  is 
considering  at  the  moment).  Unlike  long-term  memory,  the  number  of  items  that  can  be  stored  at  any  time  in  working 
memory  is  severely  limited.  (Working  memory  is  sometimes  referred  to  as  active  memory.) 
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